LICENSING NOTICE

Note that all users who use Vital DB, an open biosignal dataset, must agree to the Data Use Agreement below. If you do not agree, please close this window. The Data Use Agreement is available here: https://vitaldb.net/dataset/#h.vcpgs1yemdb5

This is the development version of the project code

For the Project Draft submission see the DL4H_Team_24_Project_Draft.ipynb notebook in the project repository.

Project repository

The project repository can be found at: https://github.com/abarrie2/cs598-dlh-project

Introduction

This project aims to reproduce findings from the paper titled "Predicting intraoperative hypotension using deep learning with waveforms of arterial blood pressure, electroencephalogram, and electrocardiogram: Retrospective study" by Jo Y-Y et al. (2022) [1]. This study introduces a deep learning model that predicts intraoperative hypotension (IOH) events before they occur, utilizing a combination of arterial blood pressure (ABP), electroencephalogram (EEG), and electrocardiogram (ECG) signals.

Background of the Problem

Intraoperative hypotension (IOH) is a common and significant surgical complication defined by a mean arterial pressure drop below 65 mmHg. It is associated with increased risks of myocardial infarction, acute kidney injury, and heightened postoperative mortality. Effective prediction and timely intervention can substantially enhance patient outcomes.

Evolution of IOH Prediction

Initial attempts to predict IOH primarily used arterial blood pressure (ABP) waveforms. A foundational study by Hatib F et al. (2018) titled "Machine-learning Algorithm to Predict Hypotension Based on High-fidelity Arterial Pressure Waveform Analysis" [2] showed that machine learning could forecast IOH events using ABP with reasonable accuracy. This finding spurred further research into utilizing various physiological signals for IOH prediction.

Subsequent advancements included the development of the Acumen™ hypotension prediction index, which was studied in "AcumenTM hypotension prediction index guidance for prevention and treatment of hypotension in noncardiac surgery: a prospective, single-arm, multicenter trial" by Bao X et al. (2024) [3]. This trial integrated a hypotension prediction index into blood pressure monitoring equipment, demonstrating its effectiveness in reducing the number and duration of IOH events during surgeries. Further study is needed to determine whether this resultant reduction in IOH events transalates into improved postoperative patient outcomes.

Current Study

Building on these advancements, the paper by Jo Y-Y et al. (2022) proposes a deep learning approach that enhances prediction accuracy by incorporating EEG and ECG signals along with ABP. This multi-modal method, evaluated over prediction windows of 3, 5, 10, and 15 minutes, aims to provide a comprehensive physiological profile that could predict IOH more accurately and earlier. Their results indicate that the combination of ABP and EEG significantly improves performance metrics such as AUROC and AUPRC, outperforming models that use fewer signals or different combinations.

Our project seeks to reproduce and verify Jo Y-Y et al.'s results to assess whether this integrated approach can indeed improve IOH prediction accuracy, thereby potentially enhancing surgical safety and patient outcomes.

Scope of Reproducibility:

The original paper investigated the following hypotheses:

1. Hypothesis 1: A model using ABP and ECG will outperform a model using ABP alone in predicting IOH.
2. Hypothesis 2: A model using ABP and EEG will outperform a model using ABP alone in predicting IOH.
3. Hypothesis 3: A model using ABP, EEG, and ECG will outperform a model using ABP alone in predicting IOH.

Results were compared using AUROC and AUPRC scores. Based on the results described in the original paper, we expect that Hypothesis 2 will be confirmed, and that Hypotheses 1 and 3 will not be confirmed.

In order to perform the corresponding experiments, we will implement a CNN-based model that can be configured to train and infer using the following four model variations:

1. ABP data alone
2. ABP and ECG data
3. ABP and EEG data
4. ABP, ECG, and EEG data

We will measure the performance of these configurations using the same AUROC and AUPRC metrics as used in the original paper. To test hypothesis 1 we will compare the AUROC and AUPRC measures between model variation 1 and model variation 2. To test hypothesis 2 we will compare the AUROC and AUPRC measures between model variation 1 and model variation 3. To test hypothesis 3 we will compare the AUROC and AUPRC measures between model variation 1 and model variation 4. For all of the above measures and experiment combinations, we will operate multiple experiments where the time-to-IOH event prediction will use the following prediction windows:

1. 3 minutes before event
2. 5 minutes before event
3. 10 minutes before event
4. 15 minutes before event

In the event that we are compute-bound, we will prioritize the 3-minute prediction window experiments as they are the most relevant to the original paper's findings.

The predictive power of ABP, ECG and ABP + ECG models at 3-, 5-, 10- and 15-minute prediction windows:

Modifications made for demo mode

In order to demonstrate the functioning of the code in a short (ie, <8 minute limit) the following options and modifications were used:

1. MAX_CASES was set to 20. The total number of cases to be used in the full training set is 3296, but the smaller numbers allows demonstration of each section of the pipeline.
2. vitaldb_cache is prepopulated in Google Colab. The cache file is approx. 800MB and contains the raw and mini-fied copies of the source dataset and is downloaded from Google Drive. This is much faster than using the vitaldb API, but is again only a fraction of the data. The full dataset can be downloaded with the API or prepopulated by following the instructions in the "Bulk Data Download" section below.
3. max_epochs is set to 6. With the small dataset, training is fast and shows the decreasing training and validation losses. In the full model run, max_epochs will be set to 100. In both cases early stopping is enabled and will stop training if the validation losses stop decreasing for five consecutive epochs.
4. Only the "ABP + EEG" combination will be run. In the final report, additional combinations will be run, as discussed later.
5. Only the 3-minute prediction window will be run. In the final report, additional prediction windows (5, 10 and 15 minutes) will be run, as discussed later.
6. No ablations are run in the demo. These will be completed for the final report.

Methodology

Methodology from Final Rubrik

• Environment
  • Python version
  • Dependencies/packages needed
• Data
  • Data download instruction
  • Data descriptions with helpful charts and visualizations
  • Preprocessing code + command
• Model
  • Citation to the original paper
  • Link to the original paper’s repo (if applicable)
  • Model descriptions
  • Implementation code
  • Pretrained model (if applicable)
• Training
  • Hyperparams
    • Report at least 3 types of hyperparameters such as learning rate, batch size, hidden size, dropout
  • Computational requirements
    • Report at least 3 types of requirements such as type of hardware, average runtime for each epoch, total number of trials, GPU hrs used, # training epochs
    • Training code
• Evaluation
  • Metrics descriptions
  • Evaluation code

The methodology section is composed of the following subsections: Environment, Data and Model.

• Environment: This section describes the setup of the environment, including the installation of necessary libraries and the configuration of the runtime environment.
• Data: This section describes the dataset used in the study, including its collection and preprocessing.
  • Data Collection: This section describes the process of downloading the dataset from VitalDB and populating the local data cache.
  • Data Preprocessing: This section describes the preprocessing steps applied to the dataset, including data selection, data cleaning, and feature extraction.
• Model: This section describes the deep learning model used in the study, including its implementation, training, and evaluation.
  • Model Implementation: This section describes the implementation of the deep learning model, including the architecture, loss function, and optimization algorithm.
  • Model Training: This section describes the training process, including the training loop, hyperparameters, and training strategy.
  • Model Evaluation: This section describes the evaluation process, including the metrics used, the evaluation strategy, and the results obtained.

Environment

Create environment

The environment setup differs based on whether you are running the code on a local machine or on Google Colab. The following sections provide instructions for setting up the environment in each case.

Local machine

Create conda environment for the project using the environment.yml file:

conda env create --prefix .envs/dlh-team24 -f environment.yml

Activate the environment with:

conda activate .envs/dlh-team24

Google Colab

The following code snippet installs the required packages and downloads the necessary files in a Google Colab environment:

# Google Colab environments have a `/content` directory. Use this as a proxy for running Colab-only code
COLAB_ENV = "google.colab" in str(get_ipython())
if COLAB_ENV:
    #install vitaldb
    %pip install vitaldb

    # Executing in Colab therefore download cached preprocessed data.
    # TODO: Integrate this with the setup local cache data section below.
    # Check for file existence before overwriting.
    import gdown
    gdown.download(id="15b5Nfhgj3McSO2GmkVUKkhSSxQXX14hJ", output="vitaldb_cache.tgz")
    !tar -zxf vitaldb_cache.tgz

    # Download sqi_filter.csv from github repo
    !wget https://raw.githubusercontent.com/abarrie2/cs598-dlh-project/main/sqi_filter.csv

All other required packages are already installed in the Google Colab environment.

Load environment

# Import packages
import os
import random
import copy
from collections import defaultdict

from timeit import default_timer as timer

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, roc_auc_score, precision_recall_curve, auc, confusion_matrix
from sklearn.metrics import RocCurveDisplay, PrecisionRecallDisplay, average_precision_score
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
import torch
from torch.utils.data import Dataset
import vitaldb
import h5py

import torch.nn as nn
import torch.nn.functional as F
from tqdm import tqdm
from datetime import datetime

Set random seeds to generate consistent results:

RANDOM_SEED = 42
random.seed(RANDOM_SEED)
np.random.seed(RANDOM_SEED)
torch.manual_seed(RANDOM_SEED)
if torch.cuda.is_available():
    torch.cuda.manual_seed(RANDOM_SEED)
    torch.cuda.manual_seed_all(RANDOM_SEED)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False
os.environ["PYTHONHASHSEED"] = str(RANDOM_SEED)

Set device to GPU or MPS if available

device = torch.device("cuda" if torch.cuda.is_available() else "mps" if (torch.backends.mps.is_available() and torch.backends.mps.is_built()) else "cpu")
print(f"Using device: {device}")

Using device: mps

Data

Data Description

Source

Data for this project is sourced from the open biosignal VitalDB dataset as described in "VitalDB, a high-fidelity multi-parameter vital signs database in surgical patients" by Lee H-C et al. (2022) [4], which contains perioperative vital signs and numerical data from 6,388 cases of non-cardiac (general, thoracic, urological, and gynecological) surgery patients who underwent routine or emergency surgery at Seoul National University Hospital between 2016 and 2017. The dataset includes ABP, ECG, and EEG signals, as well as other physiological data. The dataset is available through an API and Python library, and at PhysioNet: https://physionet.org/content/vitaldb/1.0.0/

Statistics

Characteristics of the dataset: | Characteristic | Value | Details | |-----------------------|-----------------------------|------------------------| | Total number of cases | 6,388 | | | Sex (male) | 3,243 (50.8%) | | | Age (years) | 59 | Range: 48-68 | | Height (cm) | 162 | Range: 156-169 | | Weight (kg) | 61 | Range: 53-69 | | Tram-Rac 4A tracks | 6,355 (99.5%) | Sampling rate: 500Hz | | BIS Vista tracks | 5,566 (87.1%) | Sampling rate: 128Hz | | Case duration (min) | 189 | Range: 27-1041 |

Labels are only known after processing the data. In the original paper, there were an average of 1.6 IOH events per case and 5.7 non-events per case so we expect approximately 10,221 IOH events and 364,116 non-events in the dataset.

Data Processing

Data will be processed as follows:

1. Load the dataset from VitalDB, or from a local cache if previously downloaded.
2. Apply the inclusion and exclusion selection criteria to filter the dataset according to surgery metadata.
3. Generate a minified dataset by discarding all tracks except ABP, ECG, and EEG.
4. Preprocess the data by applying band-pass and z-score normalization to the ECG and EEG signals, and filtering out ABP signals below a Signal Quality Index (SQI) threshold.
5. Generate event and non-event samples by extracting 60-second segments around IOH events and non-events.
6. Split the dataset into training, validation, and test sets with a 6:1:3 ratio, ensuring that samples from a single case are not split across different sets to avoid data leakage.

Set Up Local Data Caches

VitalDB data is static, so local copies can be stored and reused to avoid expensive downloads and to speed up data processing.

The default directory defined below is in the project .gitignore file. If this is modified, the new directory should also be added to the project .gitignore.

VITALDB_CACHE = './vitaldb_cache'
VITAL_ALL = f"{VITALDB_CACHE}/vital_all"
VITAL_MINI = f"{VITALDB_CACHE}/vital_mini"
VITAL_METADATA = f"{VITALDB_CACHE}/metadata"
VITAL_MODELS = f"{VITALDB_CACHE}/models"
VITAL_PREPROCESS_SCRATCH = f"{VITALDB_CACHE}/data_scratch"
VITAL_EXTRACTED_SEGMENTS = f"{VITALDB_CACHE}/segments"

TRACK_CACHE = None
SEGMENT_CACHE = None

# when USE_MEMORY_CACHING is enabled, track data will be persisted in an in-memory cache. Not useful once we have already pre-extracted all event segments
# DON'T USE: Stores items in memory that are later not used. Causes OOM on segment extraction.
USE_MEMORY_CACHING = False

# When RESET_CACHE is set to True, it will ensure the TRACK_CACHE is disposed and recreated when we do dataset initialization.
# Use as a shortcut to wiping cache rather than restarting kernel
RESET_CACHE = False

PREDICTION_WINDOW = 3
#PREDICTION_WINDOW = 'ALL'

ALL_PREDICTION_WINDOWS = [3, 5, 10, 15]

# Maximum number of cases of interest for which to download data.
# Set to a small value (ex: 20) for demo purposes, else set to None to disable and download and process all.
MAX_CASES = None
#MAX_CASES = 300

# Preloading Cases: when true, all matched cases will have the _mini tracks extracted and put into in-mem dict
PRELOADING_CASES = False
PRELOADING_SEGMENTS = True
# Perform Data Preprocessing: do we want to take the raw vital file and extract segments of interest for training?
PERFORM_DATA_PREPROCESSING = False

if not os.path.exists(VITALDB_CACHE):
  os.mkdir(VITALDB_CACHE)
if not os.path.exists(VITAL_ALL):
  os.mkdir(VITAL_ALL)
if not os.path.exists(VITAL_MINI):
  os.mkdir(VITAL_MINI)
if not os.path.exists(VITAL_METADATA):
  os.mkdir(VITAL_METADATA)
if not os.path.exists(VITAL_MODELS):
  os.mkdir(VITAL_MODELS)
if not os.path.exists(VITAL_PREPROCESS_SCRATCH):
  os.mkdir(VITAL_PREPROCESS_SCRATCH)
if not os.path.exists(VITAL_EXTRACTED_SEGMENTS):
  os.mkdir(VITAL_EXTRACTED_SEGMENTS)

print(os.listdir(VITALDB_CACHE))

['segments_bak', '.DS_Store', 'vital_all', 'models_all_cases_baseline', 'models', 'docs', 'segments_bak_0501_00', 'vital_mini.tar', 'data_scratch', 'osfs', 'vital_mini', 'metadata', 'segments_bak_0428_00', 'segments', 'models_old']

Bulk Data Download

This step is not required, but will significantly speed up downstream processing and avoid a high volume of API requests to the VitalDB web site.

The cache population code checks if the .vital files are locally available, and can be populated by calling the vitaldb API or by manually prepopulating the cache (recommended)

• Manually downloaded the dataset from the following site: https://physionet.org/content/vitaldb/1.0.0/
  • Download the zip file in a browser, or
  • Use wget -r -N -c -np https://physionet.org/files/vitaldb/1.0.0/ to download the files in a terminal
• Move the contents of vital_files into the ${VITAL_ALL} directory.

# Returns the Pandas DataFrame for the specified dataset.
#   One of 'cases', 'labs', or 'trks'
# If the file exists locally, create and return the DataFrame.
# Else, download and cache the csv first, then return the DataFrame.
def vitaldb_dataframe_loader(dataset_name):
    if dataset_name not in ['cases', 'labs', 'trks']:
        raise ValueError(f'Invalid dataset name: {dataset_name}')
    file_path = f'{VITAL_METADATA}/{dataset_name}.csv'
    if os.path.isfile(file_path):
        print(f'{dataset_name}.csv exists locally.')
        df = pd.read_csv(file_path)
        return df
    else:
        print(f'downloading {dataset_name} and storing in the local cache for future reuse.')
        df = pd.read_csv(f'https://api.vitaldb.net/{dataset_name}')
        df.to_csv(file_path, index=False)
        return df

Exploratory Data Analysis

Cases

cases = vitaldb_dataframe_loader('cases')
cases = cases.set_index('caseid')
cases.shape

cases.csv exists locally.

(6388, 73)

cases.index.nunique()

6388

cases.head()

	subjectid	casestart	caseend	anestart	aneend	opstart	opend	adm	dis	icu_days	...	intraop_colloid	intraop_ppf	intraop_mdz	intraop_ftn	intraop_rocu	intraop_vecu	intraop_eph	intraop_phe	intraop_epi	intraop_ca
caseid
1	5955	0	11542	-552	10848.0	1668	10368	-236220	627780	0	...	0	120	0.0	100	70	0	10	0	0	0
2	2487	0	15741	-1039	14921.0	1721	14621	-221160	1506840	0	...	0	150	0.0	0	100	0	20	0	0	0
3	2861	0	4394	-590	4210.0	1090	3010	-218640	40560	0	...	0	0	0.0	0	50	0	0	0	0	0
4	1903	0	20990	-778	20222.0	2522	17822	-201120	576480	1	...	0	80	0.0	100	100	0	50	0	0	0
5	4416	0	21531	-1009	22391.0	2591	20291	-67560	3734040	13	...	0	0	0.0	0	160	0	10	900	0	2100

5 rows × 73 columns

cases['sex'].value_counts()

sex
M    3243
F    3145
Name: count, dtype: int64

Tracks

trks = vitaldb_dataframe_loader('trks')
trks = trks.set_index('caseid')
trks.shape

trks.csv exists locally.

(486449, 2)

trks.index.nunique()

6388

trks.groupby('caseid')[['tid']].count().plot();

trks.groupby('caseid')[['tid']].count().hist();

trks.groupby('tname').count().sort_values(by='tid', ascending=False)

	tid
tname
Solar8000/HR	6387
Solar8000/PLETH_SPO2	6386
Solar8000/PLETH_HR	6386
Primus/CO2	6362
Primus/PAMB_MBAR	6361
...	...
Orchestra/AMD_VOL	1
Solar8000/ST_V5	1
Orchestra/NPS_VOL	1
Orchestra/AMD_RATE	1
Orchestra/VEC_VOL	1

196 rows × 1 columns

Parameters of Interest

Hemodynamic Parameters Reference

https://vitaldb.net/dataset/?query=overview#h.f7d712ycdpk2

SNUADC/ART

arterial blood pressure waveform

Parameter, Description, Type/Hz, Unit

SNUADC/ART, Arterial pressure wave, W/500, mmHg

trks[trks['tname'].str.contains('SNUADC/ART')].shape

(3645, 2)

SNUADC/ECG_II

electrocardiogram waveform

Parameter, Description, Type/Hz, Unit

SNUADC/ECG_II, ECG lead II wave, W/500, mV

trks[trks['tname'].str.contains('SNUADC/ECG_II')].shape

(6355, 2)

BIS/EEG1_WAV

electroencephalogram waveform

Parameter, Description, Type/Hz, Unit

BIS/EEG1_WAV, EEG wave from channel 1, W/128, uV

trks[trks['tname'].str.contains('BIS/EEG1_WAV')].shape

(5871, 2)

Cases of Interest

These are the subset of case ids for which modelling and analysis will be performed based upon inclusion criteria and waveform data availability.

# TRACK NAMES is used for metadata analysis via API
TRACK_NAMES = ['SNUADC/ART', 'SNUADC/ECG_II', 'BIS/EEG1_WAV']
TRACK_SRATES = [500, 500, 128]
# EXTRACTION TRACK NAMES adds the EVENT track which is only used when doing actual file i/o
EXTRACTION_TRACK_NAMES = ['SNUADC/ART', 'SNUADC/ECG_II', 'BIS/EEG1_WAV', 'EVENT']
EXTRACTION_TRACK_SRATES = [500, 500, 128, 1]

# As in the paper, select cases which meet the following criteria:
#
# For patients, the inclusion criteria were as follows:
# (1) adults (age >= 18)
# (2) administered general anaesthesia
# (3) undergone non-cardiac surgery. 
#
# For waveform data, the inclusion criteria were as follows:
# (1) no missing monitoring for ABP, ECG, and EEG waveforms
# (2) no cases containing false events or non-events due to poor signal quality
#     (checked in second stage of data preprocessing)

# Adult
inclusion_1 = cases.loc[cases['age'] >= 18].index
print(f'{len(cases)-len(inclusion_1)} cases excluded, {len(inclusion_1)} remaining due to age criteria')

# General Anesthesia
inclusion_2 = cases.loc[cases['ane_type'] == 'General'].index
print(f'{len(cases)-len(inclusion_2)} cases excluded, {len(inclusion_2)} remaining due to anesthesia criteria')

# Non-cardiac surgery
inclusion_3 = cases.loc[
    ~cases['opname'].str.contains("cardiac", case=False)
    & ~cases['opname'].str.contains("aneurysmal", case=False)
].index
print(f'{len(cases)-len(inclusion_3)} cases excluded, {len(inclusion_3)} remaining due to non-cardiac surgery criteria')

# ABP, ECG, EEG waveforms
inclusion_4 = trks.loc[trks['tname'].isin(TRACK_NAMES)].index.value_counts()
inclusion_4 = inclusion_4[inclusion_4 == len(TRACK_NAMES)].index
print(f'{len(cases)-len(inclusion_4)} cases excluded, {len(inclusion_4)} remaining due to missing waveform data')

# SQI filter
# NOTE: this depends on a sqi_filter.csv generated by external processing
inclusion_5 = pd.read_csv('sqi_filter.csv', header=None, names=['caseid','sqi']).set_index('caseid').index
print(f'{len(cases)-len(inclusion_5)} cases excluded, {len(inclusion_5)} remaining due to SQI threshold not being met')

# Only include cases with known good waveforms.
exclusion_6 = pd.read_csv('malformed_tracks_filter.csv', header=None, names=['caseid']).set_index('caseid').index
inclusion_6 = cases.index.difference(exclusion_6)
print(f'{len(cases)-len(inclusion_6)} cases excluded, {len(inclusion_6)} remaining due to malformed waveforms')

cases_of_interest_idx = inclusion_1 \
    .intersection(inclusion_2) \
    .intersection(inclusion_3) \
    .intersection(inclusion_4) \
    .intersection(inclusion_5) \
    .intersection(inclusion_6)

cases_of_interest = cases.loc[cases_of_interest_idx]

print()
print(f'{cases_of_interest_idx.shape[0]} out of {cases.shape[0]} total cases remaining after exclusions applied')

# Trim cases of interest to MAX_CASES
if MAX_CASES:
    cases_of_interest_idx = cases_of_interest_idx[:MAX_CASES]
print(f'{cases_of_interest_idx.shape[0]} cases of interest selected')

57 cases excluded, 6331 remaining due to age criteria
345 cases excluded, 6043 remaining due to anesthesia criteria
14 cases excluded, 6374 remaining due to non-cardiac surgery criteria
3019 cases excluded, 3369 remaining due to missing waveform data
0 cases excluded, 6388 remaining due to SQI threshold not being met
186 cases excluded, 6202 remaining due to malformed waveforms

3110 out of 6388 total cases remaining after exclusions applied
3110 cases of interest selected

cases_of_interest.head(n=5)

	subjectid	casestart	caseend	anestart	aneend	opstart	opend	adm	dis	icu_days	...	intraop_colloid	intraop_ppf	intraop_mdz	intraop_ftn	intraop_rocu	intraop_vecu	intraop_eph	intraop_phe	intraop_epi	intraop_ca
caseid
1	5955	0	11542	-552	10848.0	1668	10368	-236220	627780	0	...	0	120	0.0	100	70	0	10	0	0	0
4	1903	0	20990	-778	20222.0	2522	17822	-201120	576480	1	...	0	80	0.0	100	100	0	50	0	0	0
7	5124	0	15770	477	14817.0	3177	14577	-154320	623280	3	...	0	0	0.0	0	120	0	0	0	0	0
10	2175	0	20992	-1743	21057.0	2457	19857	-220740	3580860	1	...	0	90	0.0	0	110	0	20	500	0	600
12	491	0	31203	-220	31460.0	5360	30860	-208500	1519500	4	...	200	100	0.0	100	70	0	20	0	0	3300

5 rows × 73 columns

Tracks of Interest

These are the subset of tracks (waveforms) for the cases of interest identified above.

# A single case maps to one or more waveform tracks. Select only the tracks required for analysis.
trks_of_interest = trks.loc[cases_of_interest_idx][trks.loc[cases_of_interest_idx]['tname'].isin(TRACK_NAMES)]
trks_of_interest.shape

(9330, 2)

trks_of_interest.head(n=5)

	tname	tid
caseid
1	BIS/EEG1_WAV	0aa685df768489a18a5e9f53af0d83bf60890c73
1	SNUADC/ART	724cdd7184d7886b8f7de091c5b135bd01949959
1	SNUADC/ECG_II	8c9161aaae8cb578e2aa7b60f44234d98d2b3344
4	BIS/EEG1_WAV	1b4c2379be3397a79d3787dd810190150dc53f27
4	SNUADC/ART	e28777c4706fe3a5e714bf2d91821d22d782d802

trks_of_interest_idx = trks_of_interest.set_index('tid').index
trks_of_interest_idx.shape

(9330,)

Build Tracks Cache for Local Processing

Tracks data are large and therefore expensive to download every time used. By default, the .vital file format stores all tracks for each case internally. Since only select tracks per case are required, each .vital file can be further reduced by discarding the unused tracks.

# Ensure the full vital file dataset is available for cases of interest.
count_downloaded = 0
count_present = 0

#for i, idx in enumerate(cases.index):
for idx in cases_of_interest_idx:
    full_path = f'{VITAL_ALL}/{idx:04d}.vital'
    if not os.path.isfile(full_path):
        print(f'Missing vital file: {full_path}')
        # Download and save the file.
        vf = vitaldb.VitalFile(idx)
        vf.to_vital(full_path)
        count_downloaded += 1
    else:
        count_present += 1

print()
print(f'Count of cases of interest:           {cases_of_interest_idx.shape[0]}')
print(f'Count of vital files downloaded:      {count_downloaded}')
print(f'Count of vital files already present: {count_present}')

Count of cases of interest:           3110
Count of vital files downloaded:      0
Count of vital files already present: 3110

# Convert vital files to "mini" versions including only the subset of tracks defined in TRACK_NAMES above.
# Only perform conversion for the cases of interest.
# NOTE: If this cell is interrupted, it can be restarted and will continue where it left off.
count_minified = 0
count_present = 0
count_missing_tracks = 0
count_not_fixable = 0

vf = vitaldb.VitalFile('./vitaldb_cache/vital_all/0001.vital', EXTRACTION_TRACK_NAMES)
print(vf)

# If set to true, local mini files are checked for all tracks even if already present.
FORCE_VALIDATE = False

for idx in cases_of_interest_idx:
    full_path = f'{VITAL_ALL}/{idx:04d}.vital'
    mini_path = f'{VITAL_MINI}/{idx:04d}_mini.vital'

    if FORCE_VALIDATE or not os.path.isfile(mini_path):
        print(f'Creating mini vital file: {idx}')
        vf = vitaldb.VitalFile(full_path, EXTRACTION_TRACK_NAMES)
        
        if len(vf.get_track_names()) != 4:
            print(f'Missing track in vital file: {idx}, {set(EXTRACTION_TRACK_NAMES).difference(set(vf.get_track_names()))}')
            count_missing_tracks += 1
            
            # Attempt to download from VitalDB directly and see if missing tracks are present.
            vf = vitaldb.VitalFile(idx, EXTRACTION_TRACK_NAMES)
            
            if len(vf.get_track_names()) != 3:
                print(f'Unable to fix missing tracks: {idx}')
                count_not_fixable += 1
                continue
                
            if vf.get_track_samples(EXTRACTION_TRACK_NAMES[0], 1/EXTRACTION_TRACK_SRATES[0]).shape[0] == 0:
                print(f'Empty track: {idx}, {EXTRACTION_TRACK_NAMES[0]}')
                count_not_fixable += 1
                continue
                
            if vf.get_track_samples(EXTRACTION_TRACK_NAMES[1], 1/EXTRACTION_TRACK_SRATES[1]).shape[0] == 0:
                print(f'Empty track: {idx}, {EXTRACTION_TRACK_NAMES[1]}')
                count_not_fixable += 1
                continue
                
            if vf.get_track_samples(EXTRACTION_TRACK_NAMES[2], 1/EXTRACTION_TRACK_SRATES[2]).shape[0] == 0:
                print(f'Empty track: {idx}, {EXTRACTION_TRACK_NAMES[2]}')
                count_not_fixable += 1
                continue

            # if vf.get_track_samples(EXTRACTION_TRACK_NAMES[3], 1/EXTRACTION_TRACK_SRATES[3]).shape[0] == 0:
            #     print(f'Empty track: {idx}, {EXTRACTION_TRACK_NAMES[3]}')
            #     count_not_fixable += 1
            #     continue

        vf.to_vital(mini_path)
        count_minified += 1
    else:
        count_present += 1

print()
print(f'Count of cases of interest:           {cases_of_interest_idx.shape[0]}')
print(f'Count of vital files minified:        {count_minified}')
print(f'Count of vital files already present: {count_present}')
print(f'Count of vital files missing tracks:  {count_missing_tracks}')
print(f'Count of vital files not fixable:     {count_not_fixable}')

VitalFile('./vitaldb_cache/vital_all/0001.vital', '['EVENT', 'SNUADC/ART', 'SNUADC/ECG_II', 'BIS/EEG1_WAV']')

Count of cases of interest:           3110
Count of vital files minified:        0
Count of vital files already present: 3110
Count of vital files missing tracks:  0
Count of vital files not fixable:     0

Validate Mini Files

# Convert vital files to "mini" versions including only the subset of tracks defined in TRACK_NAMES above.
# Only perform conversion for the cases of interest.
# NOTE: If this cell is interrupted, it can be restarted and will continue where it left off.
count_missing_tracks = 0

# If true, perform fast validate that all mini files have 3 tracks.
FORCE_VALIDATE = False

if FORCE_VALIDATE:
    for idx in cases_of_interest_idx:
        mini_path = f'{VITAL_MINI}/{idx:04d}_mini.vital'

        if os.path.isfile(mini_path):
            vf = vitaldb.VitalFile(mini_path)

            if len(vf.get_track_names()) != 3:
                print(f'Missing track in vital file: {idx}, {set(TRACK_NAMES).difference(set(vf.get_track_names()))}')
                count_missing_tracks += 1

print()
print(f'Count of cases of interest:           {cases_of_interest_idx.shape[0]}')
print(f'Count of vital files missing tracks:  {count_missing_tracks}')

Count of cases of interest:           3110
Count of vital files missing tracks:  0

Filtering

Preprocessing characteristics are different for each of the three signal categories:

• ABP: no preprocessing, use as-is
• ECG: apply a 1-40Hz bandpass filter, then perform Z-score normalization
• EEG: apply a 0.5-50Hz bandpass filter

apply_bandpass_filter() implements the bandpass filter using scipy.signal

apply_zscore_normalization() implements the Z-score normalization using numpy

from scipy.signal import butter, lfilter, spectrogram

# define two methods for data preprocessing

def apply_bandpass_filter(data, lowcut, highcut, fs, order=5):
    b, a = butter(order, [lowcut, highcut], fs=fs, btype='band')
    y = lfilter(b, a, np.nan_to_num(data))
    return y

def apply_zscore_normalization(signal):
    mean = np.nanmean(signal)
    std = np.nanstd(signal)
    return (signal - mean) / std

# Filtering Demonstration

# temp experimental, code to be incorporated into overall preloader process
# for now it's just dumping example plots of the before/after filtered signal data
caseidx = 1
file_path = f"{VITAL_MINI}/{caseidx:04d}_mini.vital"
vf = vitaldb.VitalFile(file_path, TRACK_NAMES)

originalAbp = None
filteredAbp = None
originalEcg = None
filteredEcg = None
originalEeg = None
filteredEeg = None

ABP_TRACK_NAME = "SNUADC/ART"
ECG_TRACK_NAME = "SNUADC/ECG_II"
EEG_TRACK_NAME = "BIS/EEG1_WAV"

for i, (track_name, rate) in enumerate(zip(TRACK_NAMES, TRACK_SRATES)):
    # Get samples for this track
    track_samples = vf.get_track_samples(track_name, 1/rate)
    #track_samples, _ = vf.get_samples(track_name, 1/rate)
    print(f"Track {track_name} @ {rate}Hz shape {len(track_samples)}")

    if track_name == ABP_TRACK_NAME:
        # ABP waveforms are used without further pre-processing
        originalAbp = track_samples
        filteredAbp = track_samples
    elif track_name == ECG_TRACK_NAME:
        originalEcg = track_samples
        # ECG waveforms are band-pass filtered between 1 and 40 Hz, and Z-score normalized
        # first apply bandpass filter
        filteredEcg = apply_bandpass_filter(track_samples, 1, 40, rate)
        # then do z-score normalization
        filteredEcg = apply_zscore_normalization(filteredEcg)
    elif track_name == EEG_TRACK_NAME:
        # EEG waveforms are band-pass filtered between 0.5 and 50 Hz
        originalEeg = track_samples
        filteredEeg = apply_bandpass_filter(track_samples, 0.5, 50, rate, 2)

def plotSignal(data, title):
    plt.figure(figsize=(20, 5))
    plt.plot(data)
    plt.title(title)
    plt.show()

plotSignal(originalAbp, "Original ABP")
plotSignal(originalAbp, "Unfiltered ABP")
plotSignal(originalEcg, "Original ECG")
plotSignal(filteredEcg, "Filtered ECG")
plotSignal(originalEeg, "Original EEG")
plotSignal(filteredEeg, "Filtered EEG")

Track SNUADC/ART @ 500Hz shape 5770575
Track SNUADC/ECG_II @ 500Hz shape 5770575
Track BIS/EEG1_WAV @ 128Hz shape 1477268

# Preprocess data tracks
ABP_TRACK_NAME = "SNUADC/ART"
ECG_TRACK_NAME = "SNUADC/ECG_II"
EEG_TRACK_NAME = "BIS/EEG1_WAV"
EVENT_TRACK_NAME = "EVENT"
MINI_FILE_FOLDER = VITAL_MINI
CACHE_FILE_FOLDER = VITAL_PREPROCESS_SCRATCH

if RESET_CACHE:
    TRACK_CACHE = None
    SEGMENT_CACHE = None

if TRACK_CACHE is None:
    TRACK_CACHE = {}
    SEGMENT_CACHE = {}

def get_track_data(case, print_when_file_loaded = False):
    parsedFile = None
    abp = None
    eeg = None
    ecg = None
    events = None

    for i, (track_name, rate) in enumerate(zip(EXTRACTION_TRACK_NAMES, EXTRACTION_TRACK_SRATES)):
        # use integer case id and track name, delimited by pipe, as cache key
        cache_label = f"{case}|{track_name}"
        if cache_label not in TRACK_CACHE:
            if parsedFile is None:
                file_path = f"{MINI_FILE_FOLDER}/{case:04d}_mini.vital"
                if print_when_file_loaded:
                    print(f"[{datetime.now()}] Loading vital file {file_path}")
                parsedFile = vitaldb.VitalFile(file_path, EXTRACTION_TRACK_NAMES)
            dataset = np.array(parsedFile.get_track_samples(track_name, 1/rate))
            if track_name == ABP_TRACK_NAME:
                # no filtering for ABP
                abp = dataset
                abp = pd.DataFrame(abp).ffill(axis=0).bfill(axis=0)[0].values
                if USE_MEMORY_CACHING:
                    TRACK_CACHE[cache_label] = abp
            elif track_name == ECG_TRACK_NAME:
                ecg = dataset
                # apply ECG filtering: first bandpass then do z-score normalization
                ecg = pd.DataFrame(ecg).ffill(axis=0).bfill(axis=0)[0].values
                ecg = apply_bandpass_filter(ecg, 1, 40, rate, 2)
                ecg = apply_zscore_normalization(ecg)
                
                if USE_MEMORY_CACHING:
                    TRACK_CACHE[cache_label] = ecg
            elif track_name == EEG_TRACK_NAME:
                eeg = dataset
                eeg = pd.DataFrame(eeg).ffill(axis=0).bfill(axis=0)[0].values
                # apply EEG filtering: bandpass only
                eeg = apply_bandpass_filter(eeg, 0.5, 50, rate, 2)
                if USE_MEMORY_CACHING:
                    TRACK_CACHE[cache_label] = eeg
            elif track_name == EVENT_TRACK_NAME:
                events = dataset
                if USE_MEMORY_CACHING:
                    TRACK_CACHE[cache_label] = events
        else:
            # cache hit, pull from cache
            if track_name == ABP_TRACK_NAME:
                abp = TRACK_CACHE[cache_label]
            elif track_name == ECG_TRACK_NAME:
                ecg = TRACK_CACHE[cache_label]
            elif track_name == EEG_TRACK_NAME:
                eeg = TRACK_CACHE[cache_label]
            elif track_name == EVENT_TRACK_NAME:
                events = TRACK_CACHE[cache_label]

    return (abp, ecg, eeg, events)

# ABP waveforms are used without further pre-processing
# ECG waveforms are band-pass filtered between 1 and 40 Hz, and Z-score normalized
# EEG waveforms are band-pass filtered between 0.5 and 50 Hz
if PRELOADING_CASES:
    # determine disk cache file label
    maxlabel = "ALL"
    if MAX_CASES is not None:
        maxlabel = str(MAX_CASES)
    picklefile = f"{CACHE_FILE_FOLDER}/{PREDICTION_WINDOW}_minutes_MAX{maxlabel}.trackcache"

    for track in tqdm(cases_of_interest_idx):
        # getting track data will cause a cache-check and fill when missing
        # will also apply appropriate filtering per track
        get_track_data(track, False)
    
    print(f"Generated track cache, {len(TRACK_CACHE)} records generated")


def get_segment_data(file_path):
    abp = None
    eeg = None
    ecg = None

    if USE_MEMORY_CACHING:
        if file_path in SEGMENT_CACHE:
            (abp, ecg, eeg) = SEGMENT_CACHE[file_path]
            return (abp, ecg, eeg)

    try:
        with h5py.File(file_path, 'r') as f:
            abp = np.array(f['abp'])
            ecg = np.array(f['ecg'])
            eeg = np.array(f['eeg'])
        
        abp = np.array(abp)
        eeg = np.array(eeg)
        ecg = np.array(ecg)

        if len(abp) > 30000:
            abp = abp[:30000]
        elif len(ecg) < 30000:
            abp = np.resize(abp, (30000))

        if len(ecg) > 30000:
            ecg = ecg[:30000]
        elif len(ecg) < 30000:
            ecg = np.resize(ecg, (30000))

        if len(eeg) > 7680:
            eeg = eeg[:7680]
        elif len(eeg) < 7680:
            eeg = np.resize(eeg, (7680))

        if USE_MEMORY_CACHING:
            SEGMENT_CACHE[file_path] = (abp, ecg, eeg)
    except:
        abp = None
        ecg = None
        eeg = None

    return (abp, ecg, eeg)

The following method is adapted from the preprocessing block of reference [6] (https://github.com/vitaldb/examples/blob/master/hypotension_art.ipynb)

The approach first finds an interoperative hypotensive event in the ABP waveform. It then backtracks to earlier in the waveform to extract a 60 second segment representing the waveform feature to use as model input. The figure below shows an example of this approach and is reproduced from the VitalDB example notebook referenced above.

def getSurgeryBoundariesInSeconds(event, debug=False):
    eventIndices = np.argwhere(event==event)
    # we are looking for the last index where the string contains 'start
    lastStart = 0
    firstFinish = len(event)-1
    
    # find last start
    for idx in eventIndices:
        if 'started' in event[idx[0]]:
            if debug:
                print(event[idx[0]])
                print(idx[0])
            lastStart = idx[0]
    
    # find first finish
    for idx in eventIndices:
        if 'finish' in event[idx[0]]:
            if debug:
                print(event[idx[0]])
                print(idx[0])

            firstFinish = idx[0]
            break
    
    if debug:
        print(f'lastStart, firstFinish: {lastStart}, {firstFinish}')
    return (lastStart, firstFinish)

def areCaseSegmentsCached(caseid):
    seg_folder = f"{VITAL_EXTRACTED_SEGMENTS}/{caseid:04d}"
    return os.path.exists(seg_folder) and len(os.listdir(seg_folder)) > 0

def isAbpSegmentValidNumpy(samples, debug=False):
    valid = True
    if np.isnan(samples).mean() > 0.1:
        valid = False
        if debug:
            print(f">10% NaN")
    elif (samples > 200).any():
        valid = False
        if debug:
            print(f"Presence of BP > 200")
    elif (samples < 30).any():
        valid = False
        if debug:
            print(f"Presence of BP < 30")
    elif np.max(samples) - np.min(samples) < 30:
        if debug:
            print(f"Max - Min test < 30")
        valid = False
    elif (np.abs(np.diff(samples)) > 30).any():  # abrupt change -> noise
        if debug:
            print(f"Abrupt change (noise)")
        valid = False
    
    return valid

def isAbpSegmentValid(vf, debug=False):
    ABP_ECG_SRATE_HZ = 500
    ABP_TRACK_NAME = "SNUADC/ART"

    samples = np.array(vf.get_track_samples(ABP_TRACK_NAME, 1/ABP_ECG_SRATE_HZ))
    return isAbpSegmentValidNumpy(samples, debug)

def saveCaseSegments(caseid, positiveSegments, negativeSegments, compresslevel=9, debug=False, forceWrite=False):
    if len(positiveSegments) == 0 and len(negativeSegments) == 0:
        # exit early if no events found
        print(f'{caseid}: exit early, no segments to save')
        return

    # event composition
    # predictiveSegmentStart in seconds, predictiveSegmentEnd in seconds, predWindow (0 for negative), abp, ecg, eeg)
    # 0start, 1end, 2predwindow, 3abp, 4ecg, 5eeg

    seg_folder = f"{VITAL_EXTRACTED_SEGMENTS}/{caseid:04d}"
    if not os.path.exists(seg_folder):
        # if directory needs to be created, then there are no cached segments
        os.mkdir(seg_folder)
    else:
        if not forceWrite:
            # exit early if folder already exists, case already produced
            return

    # prior to writing files out, clear existing files
    for filename in os.listdir(seg_folder):
        file_path = os.path.join(seg_folder, filename)
        if debug:
            print(f'deleting: {file_path}')
        try:
            if os.path.isfile(file_path):
                os.unlink(file_path)
        except Exception as e:
            print('Failed to delete %s. Reason: %s' % (file_path, e))
    
    count_pos_saved = 0
    for i in range(0, len(positiveSegments)):
        event = positiveSegments[i]
        startIndex = event[0]
        endIndex = event[1]
        predWindow = event[2]
        abp = event[3]
        #ecg = event[4]
        #eeg = event[5]

        seg_filename = f"{caseid:04d}_{startIndex}_{predWindow:02d}_True.h5"
        seg_fullpath = f"{seg_folder}/{seg_filename}"
        if isAbpSegmentValidNumpy(abp, debug):
            count_pos_saved += 1

            abp = abp.tolist()
            ecg = event[4].tolist()
            eeg = event[5].tolist()
        
            f = h5py.File(seg_fullpath, "w")
            f.create_dataset('abp', data=abp, compression="gzip", compression_opts=compresslevel)
            f.create_dataset('ecg', data=ecg, compression="gzip", compression_opts=compresslevel)
            f.create_dataset('eeg', data=eeg, compression="gzip", compression_opts=compresslevel)
            
            f.flush()
            f.close()
            f = None

            abp = None
            ecg = None
            eeg = None

            # f.create_dataset('label', data=[1], compression="gzip", compression_opts=compresslevel)
            # f.create_dataset('pred_window', data=[event[2]], compression="gzip", compression_opts=compresslevel)
            # f.create_dataset('caseid', data=[caseid], compression="gzip", compression_opts=compresslevel)
        elif debug:
            print(f"{caseid:04d} {predWindow:02d}min {startIndex} starttime = ignored, segment validity issues")

    count_neg_saved = 0
    for i in range(0, len(negativeSegments)):
        event = negativeSegments[i]
        startIndex = event[0]
        endIndex = event[1]
        predWindow = event[2]
        abp = event[3]
        #ecg = event[4]
        #eeg = event[5]

        seg_filename = f"{caseid:04d}_{startIndex}_0_False.h5"
        seg_fullpath = f"{seg_folder}/{seg_filename}"
        if isAbpSegmentValidNumpy(abp, debug):
            count_neg_saved += 1

            abp = abp.tolist()
            ecg = event[4].tolist()
            eeg = event[5].tolist()
            
            f = h5py.File(seg_fullpath, "w")
            f.create_dataset('abp', data=abp, compression="gzip", compression_opts=compresslevel)
            f.create_dataset('ecg', data=ecg, compression="gzip", compression_opts=compresslevel)
            f.create_dataset('eeg', data=eeg, compression="gzip", compression_opts=compresslevel)
            
            f.flush()
            f.close()
            f = None

            abp = None
            ecg = None
            eeg = None

            # f.create_dataset('label', data=[0], compression="gzip", compression_opts=compresslevel)
            # f.create_dataset('pred_window', data=[0], compression="gzip", compression_opts=compresslevel)
            # f.create_dataset('caseid', data=[caseid], compression="gzip", compression_opts=compresslevel)
        elif debug:
            print(f"{caseid:04d} CleanWindow {startIndex} starttime = ignored, segment validity issues")
            
    if count_neg_saved == 0 and count_pos_saved == 0:
        print(f'{caseid}: nothing saved, all segments filtered')

# Generate hypotensive events
# Hypotensive events are defined as a 1-minute interval with sustained ABP of less than 65 mmHg
# Note: Hypotensive events should be at least 20 minutes apart to minimize potential residual effects from previous events
# Generate hypotension non-events
# To sample non-events, 30-minute segments where the ABP was above 75 mmHG were selected, and then
# three one-minute samples of each waveform were obtained from the middle of the segment
# both occur in extract_segments
#VITAL_EXTRACTED_SEGMENTS
def extract_segments(
    cases_of_interest_idx,
    debug=False,
    checkCache=True,
    forceWrite=False,
    returnSegments=False,
    skipInvalidCleanEvents=False
):
    # Sampling rate for ABP and ECG, Hz. These rates should be the same. Default = 500
    ABP_ECG_SRATE_HZ = 500

    # Sampling rate for EEG. Default = 128
    EEG_SRATE_HZ = 128

    # Final dataset for training and testing the model.
    positiveSegmentsMap = {}
    negativeSegmentsMap = {}
    iohEventsMap = {}
    cleanEventsMap = {}

    # Process each case and extract segments. For each segment identify presence of an event in the label zone.
    count_cases = len(cases_of_interest_idx)

    #for case_count, caseid in tqdm(enumerate(cases_of_interest_idx), total=count_cases):
    for case_count, caseid in enumerate(cases_of_interest_idx):
        if debug:
            print(f'Loading case: {caseid:04d}, ({case_count + 1} of {count_cases})')

        if checkCache and areCaseSegmentsCached(caseid):
            if debug:
                print(f'Skipping case: {caseid:04d}, already cached')
            # skip records we've already cached
            continue

        # read the arterial waveform
        (abp, ecg, eeg, event) = get_track_data(caseid)
        if debug:
            print(f'Length of {TRACK_NAMES[0]}:       {abp.shape[0]}')
            print(f'Length of {TRACK_NAMES[1]}:    {ecg.shape[0]}')
            print(f'Length of {TRACK_NAMES[2]}:     {eeg.shape[0]}')

        (startInSeconds, endInSeconds) = getSurgeryBoundariesInSeconds(event)
        if debug:
            print(f"Event markers indicate that surgery begins at {startInSeconds}s and ends at {endInSeconds}s.")

        track_length_seconds = int(len(abp) / ABP_ECG_SRATE_HZ)
        if debug:
            print(f"Processing case {caseid} with length {track_length_seconds}s")

        
        # check if the ABP segment in the surgery window is valid
        if debug:
            isSurgerySegmentValid = isAbpSegmentValidNumpy(abp[startInSeconds:endInSeconds])
            print(f'{caseid}: surgery segment valid: {isSurgerySegmentValid}')
        
        iohEvents = []
        cleanEvents = []
        i = 0
        started = False
        eofReached = False
        trackStartIndex = None

        # set i pointer (which operates in seconds) to start marker for surgery
        i = startInSeconds

        # FIRST PASS
        # in the first forward pass, we are going to identify the start/end boundaries of all IOH events within the case
        while i < track_length_seconds - 60 and i < endInSeconds:
            segmentStart = None
            segmentEnd = None
            segFound = False

            # look forward one minute
            abpSeg = abp[i * ABP_ECG_SRATE_HZ:(i + 60) * ABP_ECG_SRATE_HZ]

            # roll forward until we hit a one minute window where mean ABP >= 65 so we know leads are connected and it's tracking
            if not started:
                if np.nanmean(abpSeg) >= 65:
                    started = True
                    trackStartIndex = i
            # if we're started and mean abp for the window is <65, we are starting a new IOH event
            elif np.nanmean(abpSeg) < 65:
                segmentStart = i
                # now seek forward to find end of event, perpetually checking the lats minute of the IOH event
                for j in range(i + 60, track_length_seconds):
                    # look backward one minute
                    abpSegForward = abp[(j - 60) * ABP_ECG_SRATE_HZ:j * ABP_ECG_SRATE_HZ]
                    if np.nanmean(abpSegForward) >= 65:
                        segmentEnd = j - 1
                        break
                if segmentEnd is None:
                    eofReached = True
                else:
                    # otherwise, end of the IOH segment has been reached, record it
                    iohEvents.append((segmentStart, segmentEnd))
                    segFound = True
                    
                    if debug:
                        t_abp = abp[segmentStart * ABP_ECG_SRATE_HZ:segmentEnd * ABP_ECG_SRATE_HZ]
                        isIohSegmentValid = isAbpSegmentValidNumpy(t_abp)
                        print(f'{caseid}: ioh segment valid: {isIohSegmentValid}, {segmentStart}, {segmentEnd}, {t_abp.shape}')

            i += 1
            if not started:
                continue
            elif eofReached:
                break
            elif segFound:
                i = segmentEnd + 1

        # SECOND PASS
        # in the second forward pass, we are going to identify the start/end boundaries of all non-overlapping 30 minute "clean" windows
        # reuse the 'start of signal' index from our first pass
        if trackStartIndex is None:
            trackStartIndex = startInSeconds
        i = trackStartIndex
        eofReached = False

        clean_events_valid = []
        
        while i < track_length_seconds - 1800 and i < endInSeconds:
            segmentStart = None
            segmentEnd = None
            segFound = False

            startIndex = i
            endIndex = i + 1800

            # check to see if this 30 minute window overlaps any IOH events, if so ffwd to end of latest overlapping IOH
            overlapFound = False
            latestEnd = None
            for event in iohEvents:
                # case 1: starts during an event
                if startIndex >= event[0] and startIndex < event[1]:
                    latestEnd = event[1]
                    overlapFound = True
                # case 2: ends during an event
                elif endIndex >= event[0] and endIndex < event[1]:
                    latestEnd = event[1]
                    overlapFound = True
                # case 3: event occurs entirely inside of the window
                elif startIndex < event[0] and endIndex > event[1]:
                    latestEnd = event[1]
                    overlapFound = True

            # FFWD if we found an overlap
            if overlapFound:
                i = latestEnd + 1
                continue

            # look forward 30 minutes
            abpSeg = abp[startIndex * ABP_ECG_SRATE_HZ:endIndex * ABP_ECG_SRATE_HZ]

            # if we're started and mean abp for the window is >= 75, we are starting a new clean event
            if np.nanmean(abpSeg) >= 75:
                overlapFound = False
                latestEnd = None
                for event in iohEvents:
                    # case 1: starts during an event
                    if startIndex >= event[0] and startIndex < event[1]:
                        latestEnd = event[1]
                        overlapFound = True
                    # case 2: ends during an event
                    elif endIndex >= event[0] and endIndex < event[1]:
                        latestEnd = event[1]
                        overlapFound = True
                    # case 3: event occurs entirely inside of the window
                    elif startIndex < event[0] and endIndex > event[1]:
                        latestEnd = event[1]
                        overlapFound = True

                if not overlapFound:
                    segFound = True
                    segmentEnd = endIndex
                    cleanEvents.append((startIndex, endIndex))
                    
                    if skipInvalidCleanEvents:
                        isCleanSegmentValid = isAbpSegmentValidNumpy(abpSeg)
                        clean_events_valid.append(isCleanSegmentValid)
                        if debug:
                            print(f'{caseid}: clean segment valid: {isCleanSegmentValid}, {startIndex}, {endIndex}, {abpSeg.shape}')
                    else:
                        clean_events_valid.append(True)

            i += 10
            if segFound:
                i = segmentEnd + 1

        if debug:
            print(f"IOH Events for case {caseid}: {iohEvents}")
            print(f"Clean Events for case {caseid}: {cleanEvents}")

        positiveSegments = []
        negativeSegments = []

        # THIRD PASS
        # in the third pass, we will use the collections of ioh event windows to generate our actual extracted segments based on our prediction window (positive labels)
        for i in range(0, len(iohEvents)):
            if debug:
                print(f"Checking event {iohEvents[i]}")
            # we want to review current event boundaries, as well as previous event boundaries if available
            event = iohEvents[i]
            previousEvent = None
            if i > 0:
                previousEvent = iohEvents[i - 1]

            for predWindow in ALL_PREDICTION_WINDOWS:
                if debug:
                    print(f"Checking event {iohEvents[i]} for pred {predWindow}")
                iohEventStart = event[0]
                predictiveSegmentEnd = event[0] - (predWindow*60)
                predictiveSegmentStart = predictiveSegmentEnd - 60

                if (predictiveSegmentStart < 0):
                    # don't rewind before the beginning of the track
                    if debug:
                        print(f"Checking event {iohEvents[i]} for pred {predWindow} - exit, before beginning")
                    continue
                elif (predictiveSegmentStart < trackStartIndex):
                    # don't rewind before the beginning of signal in track
                    if debug:
                        print(f"Checking event {iohEvents[i]} for pred {predWindow} - exit, before track start")
                    continue
                elif previousEvent is not None:
                    # does this event window come before or during the previous event?
                    overlapFound = False
                    # case 1: starts during an event
                    if predictiveSegmentStart >= previousEvent[0] and predictiveSegmentStart < previousEvent[1]:
                        overlapFound = True
                    # case 2: ends during an event
                    elif iohEventStart >= previousEvent[0] and iohEventStart < previousEvent[1]:
                        overlapFound = True
                    # case 3: event occurs entirely inside of the window
                    elif predictiveSegmentStart < previousEvent[0] and iohEventStart > previousEvent[1]:
                        overlapFound = True
                    # do not extract a case if we overlap witha nother IOH
                    if overlapFound:
                        if debug:
                            print(f"Checking event {iohEvents[i]} for pred {predWindow} - exit, overlap with earlier segment")
                        continue

                # track the positive segment
                positiveSegments.append((predictiveSegmentStart, predictiveSegmentEnd, predWindow,
                    abp[predictiveSegmentStart*ABP_ECG_SRATE_HZ:predictiveSegmentEnd*ABP_ECG_SRATE_HZ],
                    ecg[predictiveSegmentStart*ABP_ECG_SRATE_HZ:predictiveSegmentEnd*ABP_ECG_SRATE_HZ],
                    eeg[predictiveSegmentStart*EEG_SRATE_HZ:predictiveSegmentEnd*EEG_SRATE_HZ]))

        # FOURTH PASS
        # in the fourth and final pass, we will use the collections of clean event windows to generate our actual extracted segments based (negative labels)
        for i in range(0, len(cleanEvents)):
            # Don't extract segments from invalid clean event windows.
            if not clean_events_valid[i]:
                continue
            
            # everything will be 30 minutes long at least
            event = cleanEvents[i]
            # choose sample 1 @ 10 minutes
            # choose sample 2 @ 15 minutes
            # choose sample 3 @ 20 minutes
            timeAtTen = event[0] + 600
            timeAtFifteen = event[0] + 900
            timeAtTwenty = event[0] + 1200

            negativeSegments.append((timeAtTen, timeAtTen + 60, 0,
                                   abp[timeAtTen*ABP_ECG_SRATE_HZ:(timeAtTen + 60)*ABP_ECG_SRATE_HZ],
                                   ecg[timeAtTen*ABP_ECG_SRATE_HZ:(timeAtTen + 60)*ABP_ECG_SRATE_HZ],
                                   eeg[timeAtTen*EEG_SRATE_HZ:(timeAtTen + 60)*EEG_SRATE_HZ]))
            negativeSegments.append((timeAtFifteen, timeAtFifteen + 60, 0,
                                   abp[timeAtFifteen*ABP_ECG_SRATE_HZ:(timeAtFifteen + 60)*ABP_ECG_SRATE_HZ],
                                   ecg[timeAtFifteen*ABP_ECG_SRATE_HZ:(timeAtFifteen + 60)*ABP_ECG_SRATE_HZ],
                                   eeg[timeAtFifteen*EEG_SRATE_HZ:(timeAtFifteen + 60)*EEG_SRATE_HZ]))
            negativeSegments.append((timeAtTwenty, timeAtTwenty + 60, 0,
                                   abp[timeAtTwenty*ABP_ECG_SRATE_HZ:(timeAtTwenty + 60)*ABP_ECG_SRATE_HZ],
                                   ecg[timeAtTwenty*ABP_ECG_SRATE_HZ:(timeAtTwenty + 60)*ABP_ECG_SRATE_HZ],
                                   eeg[timeAtTwenty*EEG_SRATE_HZ:(timeAtTwenty + 60)*EEG_SRATE_HZ]))

        if returnSegments:
            positiveSegmentsMap[caseid] = positiveSegments
            negativeSegmentsMap[caseid] = negativeSegments
            iohEventsMap[caseid] = iohEvents
            cleanEventsMap[caseid] = cleanEvents
        
        saveCaseSegments(caseid, positiveSegments, negativeSegments, 9, debug=debug, forceWrite=forceWrite)

        #if debug:
        print(f'{caseid}: positiveSegments: {len(positiveSegments)}, negativeSegments: {len(negativeSegments)}')

    return positiveSegmentsMap, negativeSegmentsMap, iohEventsMap, cleanEventsMap

Case Extraction - Generage Segments Needed For Training

Ensure that all needed segments are in place for the cases that are being used. If data is already stored on disk this method returns immediately.

print('Time to extract segments!')

Time to extract segments!

MANUAL_EXTRACT=True
SKIP_INVALID_CLEAN_EVENTS=True

if MANUAL_EXTRACT:
    mycoi = cases_of_interest_idx
    #mycoi = cases_of_interest_idx[:2800]
    #mycoi = [1]

    cnt = 0
    mod = 0
    for ci in mycoi:
        cnt += 1
        if mod % 100 == 0:
            print(f'count processed: {mod}, current case index: {ci}')
        try:
            p, n, i, c = extract_segments([ci], debug=False, checkCache=True, 
                                          forceWrite=True, returnSegments=False, 
                                          skipInvalidCleanEvents=SKIP_INVALID_CLEAN_EVENTS)
            p = None
            n = None
            i = None
            c = None
        except:
            print(f'error on extract segment: {ci}')
        mod += 1
    print(f'extracted: {cnt}')

count processed: 0, current case index: 1
1: positiveSegments: 12, negativeSegments: 3
4: positiveSegments: 22, negativeSegments: 3
7: positiveSegments: 12, negativeSegments: 6
10: positiveSegments: 27, negativeSegments: 6
12: positiveSegments: 22, negativeSegments: 0
13: positiveSegments: 18, negativeSegments: 0
16: positiveSegments: 12, negativeSegments: 6
19: positiveSegments: 34, negativeSegments: 3
20: positiveSegments: 17, negativeSegments: 9
22: positiveSegments: 16, negativeSegments: 12
24: positiveSegments: 3, negativeSegments: 6
25: positiveSegments: 6, negativeSegments: 12
26: exit early, no segments to save
26: positiveSegments: 0, negativeSegments: 0
27: positiveSegments: 11, negativeSegments: 12
29: positiveSegments: 8, negativeSegments: 12
31: positiveSegments: 4, negativeSegments: 3
34: positiveSegments: 4, negativeSegments: 9
38: positiveSegments: 10, negativeSegments: 0
43: positiveSegments: 14, negativeSegments: 3
44: positiveSegments: 4, negativeSegments: 6
46: positiveSegments: 0, negativeSegments: 3
49: positiveSegments: 4, negativeSegments: 0
50: positiveSegments: 12, negativeSegments: 6
52: positiveSegments: 20, negativeSegments: 3
53: positiveSegments: 0, negativeSegments: 9
55: positiveSegments: 22, negativeSegments: 6
58: positiveSegments: 14, negativeSegments: 0
59: positiveSegments: 3, negativeSegments: 0
60: positiveSegments: 7, negativeSegments: 3
61: positiveSegments: 10, negativeSegments: 3
64: positiveSegments: 16, negativeSegments: 9
65: positiveSegments: 3, negativeSegments: 3
66: positiveSegments: 14, negativeSegments: 6
67: positiveSegments: 9, negativeSegments: 0
68: positiveSegments: 0, negativeSegments: 3
69: exit early, no segments to save
69: positiveSegments: 0, negativeSegments: 0
70: exit early, no segments to save
70: positiveSegments: 0, negativeSegments: 0
74: positiveSegments: 4, negativeSegments: 6
75: positiveSegments: 42, negativeSegments: 3
77: positiveSegments: 0, negativeSegments: 9
79: positiveSegments: 14, negativeSegments: 12
83: positiveSegments: 13, negativeSegments: 0
84: positiveSegments: 4, negativeSegments: 15
87: positiveSegments: 9, negativeSegments: 0
89: positiveSegments: 5, negativeSegments: 18
92: positiveSegments: 1, negativeSegments: 0
93: positiveSegments: 7, negativeSegments: 0
94: positiveSegments: 30, negativeSegments: 6
96: positiveSegments: 33, negativeSegments: 18
97: positiveSegments: 14, negativeSegments: 0
98: positiveSegments: 8, negativeSegments: 3
101: positiveSegments: 0, negativeSegments: 9
104: positiveSegments: 15, negativeSegments: 0
105: positiveSegments: 23, negativeSegments: 0
108: positiveSegments: 5, negativeSegments: 0
110: positiveSegments: 11, negativeSegments: 0
111: positiveSegments: 11, negativeSegments: 0
112: positiveSegments: 13, negativeSegments: 0
114: positiveSegments: 8, negativeSegments: 9
116: positiveSegments: 15, negativeSegments: 0
117: positiveSegments: 15, negativeSegments: 3
118: positiveSegments: 48, negativeSegments: 0
119: positiveSegments: 2, negativeSegments: 9
124: positiveSegments: 2, negativeSegments: 3
125: positiveSegments: 4, negativeSegments: 9
126: exit early, no segments to save
126: positiveSegments: 0, negativeSegments: 0
128: positiveSegments: 7, negativeSegments: 3
130: positiveSegments: 0, negativeSegments: 9
132: positiveSegments: 0, negativeSegments: 3
135: positiveSegments: 13, negativeSegments: 0
136: positiveSegments: 10, negativeSegments: 9
137: positiveSegments: 4, negativeSegments: 0
138: positiveSegments: 0, negativeSegments: 9
139: exit early, no segments to save
139: positiveSegments: 0, negativeSegments: 0
140: positiveSegments: 12, negativeSegments: 12
142: positiveSegments: 5, negativeSegments: 9
143: positiveSegments: 12, negativeSegments: 3
145: positiveSegments: 0, negativeSegments: 6
146: positiveSegments: 14, negativeSegments: 0
148: positiveSegments: 15, negativeSegments: 0
149: positiveSegments: 8, negativeSegments: 0
152: positiveSegments: 5, negativeSegments: 0
153: positiveSegments: 12, negativeSegments: 0
156: positiveSegments: 9, negativeSegments: 12
161: positiveSegments: 12, negativeSegments: 3
163: positiveSegments: 5, negativeSegments: 0
166: positiveSegments: 18, negativeSegments: 3
167: positiveSegments: 0, negativeSegments: 3
172: exit early, no segments to save
172: positiveSegments: 0, negativeSegments: 0
175: positiveSegments: 3, negativeSegments: 3
177: positiveSegments: 12, negativeSegments: 9
178: positiveSegments: 1, negativeSegments: 12
181: positiveSegments: 8, negativeSegments: 3
183: positiveSegments: 4, negativeSegments: 0
184: positiveSegments: 46, negativeSegments: 12
186: positiveSegments: 0, negativeSegments: 3
190: positiveSegments: 4, negativeSegments: 3
191: positiveSegments: 16, negativeSegments: 0
195: positiveSegments: 21, negativeSegments: 3
197: positiveSegments: 14, negativeSegments: 6
count processed: 100, current case index: 198
198: positiveSegments: 8, negativeSegments: 12
199: exit early, no segments to save
199: positiveSegments: 0, negativeSegments: 0
200: positiveSegments: 0, negativeSegments: 6
202: positiveSegments: 0, negativeSegments: 15
203: positiveSegments: 6, negativeSegments: 0
206: positiveSegments: 8, negativeSegments: 6
208: positiveSegments: 8, negativeSegments: 0
210: positiveSegments: 4, negativeSegments: 6
218: exit early, no segments to save
218: positiveSegments: 0, negativeSegments: 0
221: exit early, no segments to save
221: positiveSegments: 0, negativeSegments: 0
222: positiveSegments: 2, negativeSegments: 3
229: positiveSegments: 0, negativeSegments: 15
232: positiveSegments: 4, negativeSegments: 3
233: positiveSegments: 14, negativeSegments: 0
234: positiveSegments: 4, negativeSegments: 9
236: positiveSegments: 13, negativeSegments: 12
237: positiveSegments: 2, negativeSegments: 9
239: positiveSegments: 0, negativeSegments: 6
241: positiveSegments: 31, negativeSegments: 12
244: positiveSegments: 14, negativeSegments: 0
247: positiveSegments: 0, negativeSegments: 18
250: positiveSegments: 0, negativeSegments: 9
251: positiveSegments: 18, negativeSegments: 0
252: positiveSegments: 17, negativeSegments: 9
256: positiveSegments: 16, negativeSegments: 3
258: exit early, no segments to save
258: positiveSegments: 0, negativeSegments: 0
261: positiveSegments: 4, negativeSegments: 6
263: positiveSegments: 0, negativeSegments: 3
266: positiveSegments: 3, negativeSegments: 9
268: positiveSegments: 2, negativeSegments: 3
269: positiveSegments: 8, negativeSegments: 6
270: positiveSegments: 2, negativeSegments: 0
279: positiveSegments: 3, negativeSegments: 6
281: positiveSegments: 5, negativeSegments: 6
282: positiveSegments: 0, negativeSegments: 3
283: positiveSegments: 14, negativeSegments: 3
286: positiveSegments: 0, negativeSegments: 9
287: positiveSegments: 4, negativeSegments: 6
293: positiveSegments: 2, negativeSegments: 3
295: positiveSegments: 13, negativeSegments: 3
296: positiveSegments: 4, negativeSegments: 6
297: positiveSegments: 0, negativeSegments: 3
300: positiveSegments: 9, negativeSegments: 3
303: positiveSegments: 11, negativeSegments: 9
304: positiveSegments: 6, negativeSegments: 9
306: positiveSegments: 8, negativeSegments: 12
308: positiveSegments: 4, negativeSegments: 6
309: positiveSegments: 13, negativeSegments: 0
312: positiveSegments: 4, negativeSegments: 6
316: positiveSegments: 7, negativeSegments: 0
318: positiveSegments: 9, negativeSegments: 3
319: exit early, no segments to save
319: positiveSegments: 0, negativeSegments: 0
321: positiveSegments: 6, negativeSegments: 9
323: positiveSegments: 12, negativeSegments: 3
327: positiveSegments: 43, negativeSegments: 0
330: positiveSegments: 4, negativeSegments: 6
337: positiveSegments: 8, negativeSegments: 3
338: exit early, no segments to save
338: positiveSegments: 0, negativeSegments: 0
342: positiveSegments: 2, negativeSegments: 6
343: positiveSegments: 3, negativeSegments: 6
345: positiveSegments: 0, negativeSegments: 15
347: exit early, no segments to save
347: positiveSegments: 0, negativeSegments: 0
348: positiveSegments: 12, negativeSegments: 3
349: positiveSegments: 8, negativeSegments: 0
353: positiveSegments: 0, negativeSegments: 12
354: positiveSegments: 22, negativeSegments: 0
355: positiveSegments: 9, negativeSegments: 3
357: positiveSegments: 4, negativeSegments: 3
358: positiveSegments: 16, negativeSegments: 0
359: positiveSegments: 37, negativeSegments: 6
362: positiveSegments: 4, negativeSegments: 12
363: positiveSegments: 16, negativeSegments: 3
367: positiveSegments: 4, negativeSegments: 3
369: positiveSegments: 4, negativeSegments: 9
370: positiveSegments: 4, negativeSegments: 0
371: positiveSegments: 4, negativeSegments: 3
375: positiveSegments: 31, negativeSegments: 0
380: positiveSegments: 4, negativeSegments: 0
382: positiveSegments: 6, negativeSegments: 15
383: positiveSegments: 12, negativeSegments: 0
384: positiveSegments: 14, negativeSegments: 0
387: positiveSegments: 3, negativeSegments: 0
388: positiveSegments: 4, negativeSegments: 6
390: positiveSegments: 13, negativeSegments: 9
397: positiveSegments: 20, negativeSegments: 0
398: positiveSegments: 0, negativeSegments: 3
402: positiveSegments: 4, negativeSegments: 6
404: positiveSegments: 0, negativeSegments: 3
405: positiveSegments: 2, negativeSegments: 0
406: positiveSegments: 4, negativeSegments: 6
408: positiveSegments: 4, negativeSegments: 0
409: positiveSegments: 16, negativeSegments: 0
413: exit early, no segments to save
413: positiveSegments: 0, negativeSegments: 0
415: positiveSegments: 9, negativeSegments: 0
416: positiveSegments: 14, negativeSegments: 0
417: positiveSegments: 9, negativeSegments: 15
418: positiveSegments: 28, negativeSegments: 0
419: positiveSegments: 3, negativeSegments: 6
425: exit early, no segments to save
425: positiveSegments: 0, negativeSegments: 0
427: positiveSegments: 0, negativeSegments: 3
count processed: 200, current case index: 431
431: positiveSegments: 9, negativeSegments: 0
435: positiveSegments: 7, negativeSegments: 6
439: positiveSegments: 7, negativeSegments: 0
440: positiveSegments: 4, negativeSegments: 6
441: positiveSegments: 4, negativeSegments: 0
442: positiveSegments: 6, negativeSegments: 0
445: positiveSegments: 12, negativeSegments: 18
447: positiveSegments: 2, negativeSegments: 3
448: positiveSegments: 4, negativeSegments: 12
449: positiveSegments: 16, negativeSegments: 0
451: positiveSegments: 21, negativeSegments: 3
452: positiveSegments: 9, negativeSegments: 3
455: positiveSegments: 4, negativeSegments: 0
458: positiveSegments: 4, negativeSegments: 3
462: positiveSegments: 9, negativeSegments: 0
466: positiveSegments: 6, negativeSegments: 12
469: positiveSegments: 34, negativeSegments: 3
472: positiveSegments: 10, negativeSegments: 21
474: positiveSegments: 28, negativeSegments: 9
476: positiveSegments: 26, negativeSegments: 3
478: positiveSegments: 5, negativeSegments: 3
481: positiveSegments: 10, negativeSegments: 3
484: positiveSegments: 0, negativeSegments: 9
485: positiveSegments: 6, negativeSegments: 0
486: positiveSegments: 14, negativeSegments: 3
488: positiveSegments: 14, negativeSegments: 0
490: positiveSegments: 7, negativeSegments: 6
492: positiveSegments: 16, negativeSegments: 12
495: positiveSegments: 4, negativeSegments: 0
499: positiveSegments: 32, negativeSegments: 6
505: positiveSegments: 23, negativeSegments: 3
512: positiveSegments: 5, negativeSegments: 18
513: positiveSegments: 13, negativeSegments: 3
516: positiveSegments: 0, negativeSegments: 6
520: positiveSegments: 10, negativeSegments: 9
521: positiveSegments: 12, negativeSegments: 0
526: positiveSegments: 7, negativeSegments: 6
527: positiveSegments: 20, negativeSegments: 0
530: positiveSegments: 7, negativeSegments: 3
531: positiveSegments: 3, negativeSegments: 0
535: positiveSegments: 0, negativeSegments: 3
536: positiveSegments: 0, negativeSegments: 3
537: positiveSegments: 4, negativeSegments: 0
541: exit early, no segments to save
541: positiveSegments: 0, negativeSegments: 0
543: positiveSegments: 13, negativeSegments: 3
544: positiveSegments: 4, negativeSegments: 6
545: positiveSegments: 2, negativeSegments: 6
547: exit early, no segments to save
547: positiveSegments: 0, negativeSegments: 0
550: positiveSegments: 11, negativeSegments: 6
551: positiveSegments: 18, negativeSegments: 3
553: positiveSegments: 30, negativeSegments: 3
559: positiveSegments: 12, negativeSegments: 6
560: positiveSegments: 0, negativeSegments: 6
561: positiveSegments: 4, negativeSegments: 0
562: positiveSegments: 6, negativeSegments: 6
563: positiveSegments: 4, negativeSegments: 0
564: positiveSegments: 8, negativeSegments: 6
566: positiveSegments: 11, negativeSegments: 6
567: positiveSegments: 0, negativeSegments: 9
568: positiveSegments: 42, negativeSegments: 0
570: positiveSegments: 8, negativeSegments: 9
573: positiveSegments: 8, negativeSegments: 6
576: positiveSegments: 4, negativeSegments: 3
577: positiveSegments: 5, negativeSegments: 9
579: positiveSegments: 4, negativeSegments: 0
582: positiveSegments: 0, negativeSegments: 3
584: nothing saved, all segments filtered
584: positiveSegments: 11, negativeSegments: 0
585: positiveSegments: 5, negativeSegments: 3
587: positiveSegments: 20, negativeSegments: 3
590: positiveSegments: 6, negativeSegments: 3
593: positiveSegments: 2, negativeSegments: 6
594: exit early, no segments to save
594: positiveSegments: 0, negativeSegments: 0
599: positiveSegments: 8, negativeSegments: 3
611: positiveSegments: 13, negativeSegments: 9
612: positiveSegments: 4, negativeSegments: 9
616: positiveSegments: 7, negativeSegments: 0
617: positiveSegments: 4, negativeSegments: 3
620: positiveSegments: 0, negativeSegments: 12
621: positiveSegments: 7, negativeSegments: 0
622: positiveSegments: 0, negativeSegments: 6
624: positiveSegments: 11, negativeSegments: 0
627: positiveSegments: 10, negativeSegments: 6
628: positiveSegments: 36, negativeSegments: 0
629: positiveSegments: 40, negativeSegments: 6
631: positiveSegments: 0, negativeSegments: 15
634: positiveSegments: 4, negativeSegments: 6
636: positiveSegments: 0, negativeSegments: 3
637: positiveSegments: 0, negativeSegments: 15
641: exit early, no segments to save
641: positiveSegments: 0, negativeSegments: 0
644: positiveSegments: 6, negativeSegments: 3
645: positiveSegments: 4, negativeSegments: 3
648: positiveSegments: 8, negativeSegments: 0
649: positiveSegments: 21, negativeSegments: 3
650: positiveSegments: 4, negativeSegments: 0
652: positiveSegments: 17, negativeSegments: 3
655: positiveSegments: 0, negativeSegments: 6
657: positiveSegments: 6, negativeSegments: 3
659: positiveSegments: 4, negativeSegments: 6
660: positiveSegments: 0, negativeSegments: 6
663: positiveSegments: 0, negativeSegments: 6
count processed: 300, current case index: 665
665: exit early, no segments to save
665: positiveSegments: 0, negativeSegments: 0
666: positiveSegments: 11, negativeSegments: 12
667: positiveSegments: 4, negativeSegments: 9
671: positiveSegments: 6, negativeSegments: 15
672: positiveSegments: 0, negativeSegments: 3
676: positiveSegments: 8, negativeSegments: 6
679: positiveSegments: 2, negativeSegments: 0
680: positiveSegments: 2, negativeSegments: 0
683: positiveSegments: 0, negativeSegments: 6
684: positiveSegments: 4, negativeSegments: 3
685: positiveSegments: 3, negativeSegments: 9
687: positiveSegments: 4, negativeSegments: 0
689: exit early, no segments to save
689: positiveSegments: 0, negativeSegments: 0
691: positiveSegments: 5, negativeSegments: 12
697: positiveSegments: 16, negativeSegments: 0
698: positiveSegments: 21, negativeSegments: 3
699: positiveSegments: 21, negativeSegments: 3
702: positiveSegments: 21, negativeSegments: 12
703: positiveSegments: 18, negativeSegments: 3
706: positiveSegments: 18, negativeSegments: 0
711: positiveSegments: 0, negativeSegments: 18
716: positiveSegments: 8, negativeSegments: 3
719: positiveSegments: 4, negativeSegments: 15
721: positiveSegments: 18, negativeSegments: 0
722: positiveSegments: 4, negativeSegments: 6
724: exit early, no segments to save
724: positiveSegments: 0, negativeSegments: 0
725: positiveSegments: 14, negativeSegments: 15
726: positiveSegments: 0, negativeSegments: 3
728: positiveSegments: 25, negativeSegments: 15
729: positiveSegments: 4, negativeSegments: 3
730: positiveSegments: 9, negativeSegments: 3
733: positiveSegments: 12, negativeSegments: 3
734: positiveSegments: 1, negativeSegments: 15
737: positiveSegments: 4, negativeSegments: 0
739: positiveSegments: 0, negativeSegments: 3
740: positiveSegments: 6, negativeSegments: 3
742: positiveSegments: 4, negativeSegments: 6
744: positiveSegments: 0, negativeSegments: 3
745: exit early, no segments to save
745: positiveSegments: 0, negativeSegments: 0
746: positiveSegments: 7, negativeSegments: 3
747: exit early, no segments to save
747: positiveSegments: 0, negativeSegments: 0
748: positiveSegments: 20, negativeSegments: 12
749: positiveSegments: 2, negativeSegments: 0
750: positiveSegments: 59, negativeSegments: 12
751: positiveSegments: 5, negativeSegments: 15
752: positiveSegments: 2, negativeSegments: 12
753: positiveSegments: 4, negativeSegments: 0
755: positiveSegments: 8, negativeSegments: 3
756: positiveSegments: 4, negativeSegments: 9
757: positiveSegments: 2, negativeSegments: 6
758: positiveSegments: 11, negativeSegments: 0
761: positiveSegments: 4, negativeSegments: 0
762: positiveSegments: 7, negativeSegments: 9
763: positiveSegments: 21, negativeSegments: 6
764: positiveSegments: 26, negativeSegments: 6
765: positiveSegments: 25, negativeSegments: 9
767: positiveSegments: 4, negativeSegments: 6
768: positiveSegments: 7, negativeSegments: 0
770: positiveSegments: 5, negativeSegments: 0
772: positiveSegments: 0, negativeSegments: 3
773: positiveSegments: 0, negativeSegments: 3
774: positiveSegments: 22, negativeSegments: 3
775: positiveSegments: 22, negativeSegments: 0
776: positiveSegments: 12, negativeSegments: 3
777: positiveSegments: 23, negativeSegments: 3
779: positiveSegments: 5, negativeSegments: 12
781: positiveSegments: 7, negativeSegments: 0
783: positiveSegments: 4, negativeSegments: 0
788: positiveSegments: 4, negativeSegments: 3
792: positiveSegments: 6, negativeSegments: 0
793: positiveSegments: 21, negativeSegments: 0
794: positiveSegments: 13, negativeSegments: 0
795: positiveSegments: 9, negativeSegments: 9
797: exit early, no segments to save
797: positiveSegments: 0, negativeSegments: 0
800: positiveSegments: 15, negativeSegments: 3
802: positiveSegments: 24, negativeSegments: 6
807: positiveSegments: 8, negativeSegments: 6
808: positiveSegments: 14, negativeSegments: 3
810: positiveSegments: 2, negativeSegments: 0
812: positiveSegments: 4, negativeSegments: 3
813: positiveSegments: 9, negativeSegments: 3
814: positiveSegments: 7, negativeSegments: 12
815: positiveSegments: 0, negativeSegments: 18
816: positiveSegments: 4, negativeSegments: 12
818: exit early, no segments to save
818: positiveSegments: 0, negativeSegments: 0
819: positiveSegments: 14, negativeSegments: 0
822: positiveSegments: 18, negativeSegments: 15
825: positiveSegments: 0, negativeSegments: 9
827: positiveSegments: 4, negativeSegments: 0
830: positiveSegments: 2, negativeSegments: 6
831: positiveSegments: 0, negativeSegments: 6
833: positiveSegments: 13, negativeSegments: 3
835: positiveSegments: 0, negativeSegments: 3
841: positiveSegments: 0, negativeSegments: 9
843: positiveSegments: 28, negativeSegments: 9
846: positiveSegments: 14, negativeSegments: 0
847: positiveSegments: 0, negativeSegments: 12
848: positiveSegments: 4, negativeSegments: 0
851: positiveSegments: 25, negativeSegments: 3
852: positiveSegments: 1, negativeSegments: 3
count processed: 400, current case index: 853
853: positiveSegments: 4, negativeSegments: 6
855: positiveSegments: 8, negativeSegments: 3
859: positiveSegments: 8, negativeSegments: 9
860: positiveSegments: 12, negativeSegments: 3
864: positiveSegments: 11, negativeSegments: 0
865: positiveSegments: 11, negativeSegments: 6
866: positiveSegments: 1, negativeSegments: 6
868: positiveSegments: 10, negativeSegments: 0
869: positiveSegments: 16, negativeSegments: 12
870: positiveSegments: 19, negativeSegments: 3
871: positiveSegments: 2, negativeSegments: 6
872: positiveSegments: 0, negativeSegments: 6
876: positiveSegments: 2, negativeSegments: 9
879: positiveSegments: 5, negativeSegments: 3
880: positiveSegments: 8, negativeSegments: 3
881: exit early, no segments to save
881: positiveSegments: 0, negativeSegments: 0
883: positiveSegments: 21, negativeSegments: 6
885: positiveSegments: 4, negativeSegments: 15
886: positiveSegments: 14, negativeSegments: 18
887: positiveSegments: 4, negativeSegments: 3
890: positiveSegments: 6, negativeSegments: 0
892: positiveSegments: 10, negativeSegments: 3
894: positiveSegments: 4, negativeSegments: 0
898: positiveSegments: 0, negativeSegments: 3
907: positiveSegments: 13, negativeSegments: 9
912: positiveSegments: 9, negativeSegments: 3
913: positiveSegments: 5, negativeSegments: 3
916: positiveSegments: 19, negativeSegments: 6
917: positiveSegments: 4, negativeSegments: 3
919: positiveSegments: 0, negativeSegments: 6
922: positiveSegments: 3, negativeSegments: 3
925: exit early, no segments to save
925: positiveSegments: 0, negativeSegments: 0
926: positiveSegments: 5, negativeSegments: 0
931: positiveSegments: 5, negativeSegments: 0
932: positiveSegments: 10, negativeSegments: 3
933: exit early, no segments to save
933: positiveSegments: 0, negativeSegments: 0
936: positiveSegments: 9, negativeSegments: 3
937: positiveSegments: 5, negativeSegments: 12
938: positiveSegments: 8, negativeSegments: 0
939: positiveSegments: 12, negativeSegments: 3
940: positiveSegments: 8, negativeSegments: 0
944: positiveSegments: 16, negativeSegments: 12
945: positiveSegments: 26, negativeSegments: 3
946: positiveSegments: 12, negativeSegments: 0
947: positiveSegments: 6, negativeSegments: 15
948: positiveSegments: 11, negativeSegments: 6
949: positiveSegments: 10, negativeSegments: 3
952: exit early, no segments to save
952: positiveSegments: 0, negativeSegments: 0
954: positiveSegments: 4, negativeSegments: 6
957: positiveSegments: 4, negativeSegments: 9
958: positiveSegments: 7, negativeSegments: 0
959: positiveSegments: 0, negativeSegments: 9
963: positiveSegments: 8, negativeSegments: 12
967: positiveSegments: 0, negativeSegments: 3
969: positiveSegments: 6, negativeSegments: 0
971: positiveSegments: 7, negativeSegments: 9
972: positiveSegments: 12, negativeSegments: 0
973: positiveSegments: 6, negativeSegments: 0
976: positiveSegments: 0, negativeSegments: 15
977: positiveSegments: 6, negativeSegments: 3
979: positiveSegments: 10, negativeSegments: 0
980: positiveSegments: 12, negativeSegments: 15
983: positiveSegments: 0, negativeSegments: 9
984: positiveSegments: 8, negativeSegments: 0
985: positiveSegments: 16, negativeSegments: 6
986: positiveSegments: 0, negativeSegments: 9
988: positiveSegments: 5, negativeSegments: 9
990: positiveSegments: 13, negativeSegments: 0
991: positiveSegments: 11, negativeSegments: 0
992: positiveSegments: 4, negativeSegments: 12
994: positiveSegments: 6, negativeSegments: 0
995: positiveSegments: 12, negativeSegments: 3
1002: positiveSegments: 8, negativeSegments: 6
1003: positiveSegments: 2, negativeSegments: 3
1005: positiveSegments: 4, negativeSegments: 12
1012: positiveSegments: 9, negativeSegments: 9
1013: positiveSegments: 19, negativeSegments: 9
1015: positiveSegments: 1, negativeSegments: 0
1016: positiveSegments: 5, negativeSegments: 6
1017: positiveSegments: 3, negativeSegments: 3
1018: positiveSegments: 16, negativeSegments: 12
1020: positiveSegments: 4, negativeSegments: 6
1022: positiveSegments: 0, negativeSegments: 9
1024: positiveSegments: 6, negativeSegments: 0
1025: positiveSegments: 10, negativeSegments: 15
1026: positiveSegments: 16, negativeSegments: 12
1027: positiveSegments: 16, negativeSegments: 3
1028: positiveSegments: 0, negativeSegments: 3
1029: positiveSegments: 2, negativeSegments: 9
1030: positiveSegments: 2, negativeSegments: 6
1032: positiveSegments: 11, negativeSegments: 0
1033: positiveSegments: 4, negativeSegments: 3
1034: exit early, no segments to save
1034: positiveSegments: 0, negativeSegments: 0
1035: positiveSegments: 2, negativeSegments: 9
1037: positiveSegments: 1, negativeSegments: 21
1038: positiveSegments: 0, negativeSegments: 6
1040: positiveSegments: 8, negativeSegments: 3
1041: positiveSegments: 4, negativeSegments: 6
1043: positiveSegments: 0, negativeSegments: 3
1044: positiveSegments: 8, negativeSegments: 9
count processed: 500, current case index: 1046
1046: positiveSegments: 11, negativeSegments: 0
1047: positiveSegments: 0, negativeSegments: 6
1049: positiveSegments: 4, negativeSegments: 6
1050: positiveSegments: 7, negativeSegments: 3
1051: positiveSegments: 4, negativeSegments: 3
1055: exit early, no segments to save
1055: positiveSegments: 0, negativeSegments: 0
1056: positiveSegments: 10, negativeSegments: 0
1061: positiveSegments: 0, negativeSegments: 6
1063: positiveSegments: 14, negativeSegments: 3
1065: positiveSegments: 8, negativeSegments: 0
1069: positiveSegments: 32, negativeSegments: 3
1073: positiveSegments: 11, negativeSegments: 0
1074: positiveSegments: 13, negativeSegments: 0
1076: positiveSegments: 18, negativeSegments: 3
1077: positiveSegments: 4, negativeSegments: 0
1078: positiveSegments: 8, negativeSegments: 3
1081: positiveSegments: 7, negativeSegments: 0
1083: positiveSegments: 10, negativeSegments: 0
1084: positiveSegments: 4, negativeSegments: 6
1086: positiveSegments: 21, negativeSegments: 6
1087: positiveSegments: 13, negativeSegments: 6
1088: positiveSegments: 0, negativeSegments: 6
1089: positiveSegments: 4, negativeSegments: 3
1090: positiveSegments: 10, negativeSegments: 3
1091: positiveSegments: 1, negativeSegments: 0
1093: positiveSegments: 12, negativeSegments: 6
1094: positiveSegments: 17, negativeSegments: 18
1095: positiveSegments: 16, negativeSegments: 18
1096: positiveSegments: 6, negativeSegments: 12
1097: positiveSegments: 14, negativeSegments: 3
1098: positiveSegments: 8, negativeSegments: 0
1102: positiveSegments: 7, negativeSegments: 0
1108: positiveSegments: 8, negativeSegments: 3
1109: positiveSegments: 0, negativeSegments: 3
1113: positiveSegments: 8, negativeSegments: 12
1114: positiveSegments: 20, negativeSegments: 0
1115: positiveSegments: 10, negativeSegments: 12
1118: positiveSegments: 16, negativeSegments: 12
1123: positiveSegments: 17, negativeSegments: 6
1124: positiveSegments: 9, negativeSegments: 0
1125: positiveSegments: 12, negativeSegments: 12
1127: positiveSegments: 10, negativeSegments: 3
1131: positiveSegments: 0, negativeSegments: 6
1132: positiveSegments: 27, negativeSegments: 6
1133: positiveSegments: 3, negativeSegments: 0
1135: positiveSegments: 0, negativeSegments: 6
1138: positiveSegments: 8, negativeSegments: 0
1139: positiveSegments: 0, negativeSegments: 3
1143: positiveSegments: 7, negativeSegments: 0
1144: positiveSegments: 5, negativeSegments: 0
1145: positiveSegments: 4, negativeSegments: 12
1154: positiveSegments: 0, negativeSegments: 6
1156: exit early, no segments to save
1156: positiveSegments: 0, negativeSegments: 0
1158: positiveSegments: 4, negativeSegments: 0
1159: positiveSegments: 10, negativeSegments: 3
1160: positiveSegments: 17, negativeSegments: 0
1165: positiveSegments: 22, negativeSegments: 0
1166: positiveSegments: 15, negativeSegments: 3
1170: positiveSegments: 0, negativeSegments: 6
1174: positiveSegments: 4, negativeSegments: 0
1176: positiveSegments: 0, negativeSegments: 6
1180: positiveSegments: 10, negativeSegments: 9
1181: positiveSegments: 12, negativeSegments: 0
1182: positiveSegments: 8, negativeSegments: 9
1184: positiveSegments: 10, negativeSegments: 0
1185: positiveSegments: 35, negativeSegments: 6
1186: positiveSegments: 0, negativeSegments: 3
1187: positiveSegments: 2, negativeSegments: 3
1189: positiveSegments: 17, negativeSegments: 0
1190: exit early, no segments to save
1190: positiveSegments: 0, negativeSegments: 0
1191: positiveSegments: 47, negativeSegments: 6
1193: positiveSegments: 26, negativeSegments: 3
1194: positiveSegments: 20, negativeSegments: 3
1196: positiveSegments: 4, negativeSegments: 0
1199: positiveSegments: 4, negativeSegments: 3
1200: positiveSegments: 11, negativeSegments: 6
1201: positiveSegments: 0, negativeSegments: 6
1202: positiveSegments: 0, negativeSegments: 3
1203: positiveSegments: 2, negativeSegments: 0
1204: positiveSegments: 19, negativeSegments: 3
1205: positiveSegments: 14, negativeSegments: 3
1208: positiveSegments: 8, negativeSegments: 0
1209: positiveSegments: 7, negativeSegments: 0
1212: positiveSegments: 0, negativeSegments: 3
1213: positiveSegments: 1, negativeSegments: 6
1215: positiveSegments: 28, negativeSegments: 3
1216: positiveSegments: 21, negativeSegments: 21
1217: positiveSegments: 9, negativeSegments: 0
1219: positiveSegments: 16, negativeSegments: 6
1221: positiveSegments: 10, negativeSegments: 6
1222: positiveSegments: 4, negativeSegments: 9
1224: positiveSegments: 12, negativeSegments: 0
1225: positiveSegments: 8, negativeSegments: 3
1228: positiveSegments: 15, negativeSegments: 0
1229: positiveSegments: 26, negativeSegments: 0
1230: positiveSegments: 14, negativeSegments: 15
1231: positiveSegments: 3, negativeSegments: 0
1232: positiveSegments: 3, negativeSegments: 6
1233: positiveSegments: 9, negativeSegments: 0
1234: positiveSegments: 0, negativeSegments: 3
count processed: 600, current case index: 1236
1236: positiveSegments: 16, negativeSegments: 0
1237: positiveSegments: 6, negativeSegments: 0
1238: exit early, no segments to save
1238: positiveSegments: 0, negativeSegments: 0
1239: positiveSegments: 2, negativeSegments: 12
1240: positiveSegments: 8, negativeSegments: 6
1244: positiveSegments: 11, negativeSegments: 12
1248: positiveSegments: 4, negativeSegments: 12
1249: positiveSegments: 5, negativeSegments: 6
1256: positiveSegments: 0, negativeSegments: 6
1261: positiveSegments: 14, negativeSegments: 6
1263: positiveSegments: 11, negativeSegments: 9
1264: exit early, no segments to save
1264: positiveSegments: 0, negativeSegments: 0
1265: positiveSegments: 0, negativeSegments: 9
1267: positiveSegments: 0, negativeSegments: 6
1268: positiveSegments: 0, negativeSegments: 6
1271: exit early, no segments to save
1271: positiveSegments: 0, negativeSegments: 0
1272: positiveSegments: 15, negativeSegments: 0
1275: positiveSegments: 26, negativeSegments: 3
1277: positiveSegments: 4, negativeSegments: 9
1279: positiveSegments: 0, negativeSegments: 6
1280: positiveSegments: 0, negativeSegments: 6
1283: positiveSegments: 5, negativeSegments: 0
1285: positiveSegments: 7, negativeSegments: 6
1286: positiveSegments: 27, negativeSegments: 15
1290: positiveSegments: 0, negativeSegments: 3
1291: positiveSegments: 19, negativeSegments: 6
1292: positiveSegments: 25, negativeSegments: 6
1293: positiveSegments: 17, negativeSegments: 0
1294: positiveSegments: 4, negativeSegments: 0
1297: positiveSegments: 7, negativeSegments: 0
1298: positiveSegments: 4, negativeSegments: 6
1300: positiveSegments: 9, negativeSegments: 0
1301: exit early, no segments to save
1301: positiveSegments: 0, negativeSegments: 0
1302: positiveSegments: 0, negativeSegments: 12
1303: positiveSegments: 4, negativeSegments: 9
1305: positiveSegments: 18, negativeSegments: 0
1307: positiveSegments: 14, negativeSegments: 12
1309: positiveSegments: 13, negativeSegments: 9
1311: positiveSegments: 0, negativeSegments: 3
1313: positiveSegments: 11, negativeSegments: 6
1315: positiveSegments: 10, negativeSegments: 6
1316: positiveSegments: 14, negativeSegments: 3
1317: positiveSegments: 4, negativeSegments: 0
1319: positiveSegments: 4, negativeSegments: 3
1320: positiveSegments: 6, negativeSegments: 0
1321: positiveSegments: 2, negativeSegments: 15
1323: positiveSegments: 8, negativeSegments: 0
1324: positiveSegments: 6, negativeSegments: 9
1325: positiveSegments: 20, negativeSegments: 9
1333: positiveSegments: 2, negativeSegments: 6
1335: positiveSegments: 22, negativeSegments: 9
1339: positiveSegments: 2, negativeSegments: 3
1340: positiveSegments: 0, negativeSegments: 3
1341: positiveSegments: 4, negativeSegments: 3
1343: positiveSegments: 10, negativeSegments: 3
1344: positiveSegments: 0, negativeSegments: 9
1346: positiveSegments: 0, negativeSegments: 9
1347: positiveSegments: 0, negativeSegments: 9
1350: positiveSegments: 2, negativeSegments: 15
1353: positiveSegments: 9, negativeSegments: 0
1356: positiveSegments: 7, negativeSegments: 0
1358: positiveSegments: 3, negativeSegments: 9
1359: positiveSegments: 27, negativeSegments: 6
1362: positiveSegments: 9, negativeSegments: 3
1364: positiveSegments: 18, negativeSegments: 0
1365: positiveSegments: 8, negativeSegments: 0
1367: positiveSegments: 23, negativeSegments: 3
1368: positiveSegments: 4, negativeSegments: 0
1369: exit early, no segments to save
1369: positiveSegments: 0, negativeSegments: 0
1374: positiveSegments: 20, negativeSegments: 15
1375: positiveSegments: 11, negativeSegments: 3
1376: positiveSegments: 4, negativeSegments: 3
1381: positiveSegments: 13, negativeSegments: 6
1383: positiveSegments: 10, negativeSegments: 9
1386: positiveSegments: 0, negativeSegments: 3
1389: positiveSegments: 4, negativeSegments: 9
1392: positiveSegments: 3, negativeSegments: 0
1396: positiveSegments: 8, negativeSegments: 9
1397: positiveSegments: 13, negativeSegments: 3
1398: positiveSegments: 3, negativeSegments: 6
1399: positiveSegments: 19, negativeSegments: 3
1402: positiveSegments: 14, negativeSegments: 0
1403: positiveSegments: 22, negativeSegments: 24
1404: positiveSegments: 0, negativeSegments: 3
1407: positiveSegments: 36, negativeSegments: 3
1408: positiveSegments: 0, negativeSegments: 3
1413: exit early, no segments to save
1413: positiveSegments: 0, negativeSegments: 0
1414: positiveSegments: 10, negativeSegments: 0
1415: positiveSegments: 6, negativeSegments: 3
1416: positiveSegments: 23, negativeSegments: 3
1417: positiveSegments: 3, negativeSegments: 3
1421: positiveSegments: 8, negativeSegments: 6
1422: positiveSegments: 2, negativeSegments: 15
1426: positiveSegments: 4, negativeSegments: 6
1428: positiveSegments: 5, negativeSegments: 6
1432: positiveSegments: 16, negativeSegments: 6
1434: positiveSegments: 24, negativeSegments: 3
1436: positiveSegments: 8, negativeSegments: 12
1438: positiveSegments: 9, negativeSegments: 0
1439: positiveSegments: 10, negativeSegments: 3
count processed: 700, current case index: 1440
1440: exit early, no segments to save
1440: positiveSegments: 0, negativeSegments: 0
1442: positiveSegments: 6, negativeSegments: 12
1444: positiveSegments: 4, negativeSegments: 0
1446: positiveSegments: 9, negativeSegments: 3
1451: exit early, no segments to save
1451: positiveSegments: 0, negativeSegments: 0
1452: positiveSegments: 0, negativeSegments: 6
1454: positiveSegments: 0, negativeSegments: 18
1456: positiveSegments: 4, negativeSegments: 0
1458: positiveSegments: 12, negativeSegments: 0
1463: positiveSegments: 2, negativeSegments: 9
1465: positiveSegments: 8, negativeSegments: 3
1468: positiveSegments: 11, negativeSegments: 18
1469: positiveSegments: 30, negativeSegments: 3
1470: positiveSegments: 7, negativeSegments: 0
1471: positiveSegments: 4, negativeSegments: 0
1473: positiveSegments: 4, negativeSegments: 6
1474: positiveSegments: 3, negativeSegments: 6
1475: positiveSegments: 41, negativeSegments: 3
1478: positiveSegments: 7, negativeSegments: 6
1479: positiveSegments: 4, negativeSegments: 3
1482: positiveSegments: 0, negativeSegments: 18
1485: positiveSegments: 26, negativeSegments: 0
1486: positiveSegments: 22, negativeSegments: 0
1487: positiveSegments: 0, negativeSegments: 6
1488: positiveSegments: 12, negativeSegments: 0
1489: positiveSegments: 15, negativeSegments: 3
1490: positiveSegments: 11, negativeSegments: 3
1492: positiveSegments: 43, negativeSegments: 0
1493: positiveSegments: 13, negativeSegments: 9
1496: positiveSegments: 4, negativeSegments: 6
1497: positiveSegments: 16, negativeSegments: 6
1498: positiveSegments: 0, negativeSegments: 6
1500: positiveSegments: 5, negativeSegments: 6
1503: positiveSegments: 7, negativeSegments: 0
1505: exit early, no segments to save
1505: positiveSegments: 0, negativeSegments: 0
1512: positiveSegments: 4, negativeSegments: 3
1515: positiveSegments: 38, negativeSegments: 3
1520: positiveSegments: 18, negativeSegments: 0
1521: positiveSegments: 16, negativeSegments: 9
1522: positiveSegments: 4, negativeSegments: 9
1523: positiveSegments: 11, negativeSegments: 27
1525: positiveSegments: 22, negativeSegments: 3
1526: positiveSegments: 0, negativeSegments: 9
1527: positiveSegments: 0, negativeSegments: 6
1536: positiveSegments: 0, negativeSegments: 6
1537: positiveSegments: 12, negativeSegments: 0
1539: positiveSegments: 7, negativeSegments: 24
1540: positiveSegments: 6, negativeSegments: 0
1541: positiveSegments: 13, negativeSegments: 0
1542: positiveSegments: 16, negativeSegments: 6
1545: positiveSegments: 19, negativeSegments: 3
1546: positiveSegments: 4, negativeSegments: 3
1548: positiveSegments: 0, negativeSegments: 12
1549: positiveSegments: 7, negativeSegments: 6
1552: positiveSegments: 5, negativeSegments: 0
1553: positiveSegments: 5, negativeSegments: 0
1554: exit early, no segments to save
1554: positiveSegments: 0, negativeSegments: 0
1555: exit early, no segments to save
1555: positiveSegments: 0, negativeSegments: 0
1556: positiveSegments: 5, negativeSegments: 18
1558: positiveSegments: 18, negativeSegments: 0
1559: positiveSegments: 14, negativeSegments: 6
1561: positiveSegments: 11, negativeSegments: 3
1562: positiveSegments: 6, negativeSegments: 9
1563: positiveSegments: 4, negativeSegments: 0
1564: positiveSegments: 30, negativeSegments: 3
1566: positiveSegments: 8, negativeSegments: 6
1567: positiveSegments: 11, negativeSegments: 0
1568: positiveSegments: 16, negativeSegments: 0
1574: positiveSegments: 4, negativeSegments: 12
1575: positiveSegments: 8, negativeSegments: 6
1580: positiveSegments: 20, negativeSegments: 3
1581: positiveSegments: 4, negativeSegments: 9
1583: positiveSegments: 20, negativeSegments: 6
1585: exit early, no segments to save
1585: positiveSegments: 0, negativeSegments: 0
1586: positiveSegments: 4, negativeSegments: 9
1590: positiveSegments: 20, negativeSegments: 0
1591: positiveSegments: 9, negativeSegments: 6
1594: positiveSegments: 4, negativeSegments: 0
1595: positiveSegments: 8, negativeSegments: 12
1596: exit early, no segments to save
1596: positiveSegments: 0, negativeSegments: 0
1597: positiveSegments: 8, negativeSegments: 18
1599: positiveSegments: 13, negativeSegments: 15
1600: positiveSegments: 4, negativeSegments: 0
1602: positiveSegments: 6, negativeSegments: 18
1605: positiveSegments: 17, negativeSegments: 12
1608: positiveSegments: 8, negativeSegments: 0
1610: positiveSegments: 0, negativeSegments: 3
1611: exit early, no segments to save
1611: positiveSegments: 0, negativeSegments: 0
1612: exit early, no segments to save
1612: positiveSegments: 0, negativeSegments: 0
1613: positiveSegments: 4, negativeSegments: 6
1614: positiveSegments: 0, negativeSegments: 9
1615: positiveSegments: 11, negativeSegments: 12
1616: positiveSegments: 12, negativeSegments: 6
1618: positiveSegments: 4, negativeSegments: 0
1620: positiveSegments: 14, negativeSegments: 0
1623: positiveSegments: 6, negativeSegments: 0
1630: positiveSegments: 15, negativeSegments: 3
1632: positiveSegments: 7, negativeSegments: 0
1633: positiveSegments: 18, negativeSegments: 3
1636: positiveSegments: 7, negativeSegments: 3
count processed: 800, current case index: 1639
1639: positiveSegments: 13, negativeSegments: 0
1641: positiveSegments: 13, negativeSegments: 9
1642: positiveSegments: 9, negativeSegments: 0
1647: positiveSegments: 8, negativeSegments: 3
1648: positiveSegments: 10, negativeSegments: 3
1656: positiveSegments: 0, negativeSegments: 3
1657: positiveSegments: 0, negativeSegments: 12
1658: positiveSegments: 8, negativeSegments: 6
1662: exit early, no segments to save
1662: positiveSegments: 0, negativeSegments: 0
1665: positiveSegments: 15, negativeSegments: 0
1666: positiveSegments: 12, negativeSegments: 9
1668: positiveSegments: 4, negativeSegments: 0
1671: positiveSegments: 19, negativeSegments: 3
1672: positiveSegments: 4, negativeSegments: 9
1673: positiveSegments: 10, negativeSegments: 0
1674: positiveSegments: 18, negativeSegments: 0
1675: nothing saved, all segments filtered
1675: positiveSegments: 3, negativeSegments: 0
1676: exit early, no segments to save
1676: positiveSegments: 0, negativeSegments: 0
1681: positiveSegments: 4, negativeSegments: 0
1684: positiveSegments: 8, negativeSegments: 0
1685: positiveSegments: 0, negativeSegments: 3
1687: positiveSegments: 0, negativeSegments: 3
1688: positiveSegments: 6, negativeSegments: 0
1689: positiveSegments: 3, negativeSegments: 6
1690: positiveSegments: 10, negativeSegments: 6
1694: positiveSegments: 15, negativeSegments: 3
1695: positiveSegments: 4, negativeSegments: 3
1696: positiveSegments: 0, negativeSegments: 18
1697: positiveSegments: 3, negativeSegments: 0
1699: positiveSegments: 0, negativeSegments: 3
1700: positiveSegments: 10, negativeSegments: 6
1703: positiveSegments: 38, negativeSegments: 12
1705: positiveSegments: 4, negativeSegments: 12
1706: positiveSegments: 3, negativeSegments: 6
1708: positiveSegments: 17, negativeSegments: 12
1710: positiveSegments: 14, negativeSegments: 3
1714: positiveSegments: 6, negativeSegments: 9
1716: positiveSegments: 23, negativeSegments: 0
1718: positiveSegments: 27, negativeSegments: 3
1719: positiveSegments: 4, negativeSegments: 0
1722: positiveSegments: 0, negativeSegments: 18
1724: positiveSegments: 9, negativeSegments: 3
1726: positiveSegments: 6, negativeSegments: 9
1728: positiveSegments: 0, negativeSegments: 9
1729: positiveSegments: 16, negativeSegments: 15
1730: positiveSegments: 14, negativeSegments: 0
1732: positiveSegments: 8, negativeSegments: 6
1733: positiveSegments: 18, negativeSegments: 0
1735: positiveSegments: 4, negativeSegments: 6
1737: positiveSegments: 12, negativeSegments: 6
1738: positiveSegments: 19, negativeSegments: 0
1743: positiveSegments: 17, negativeSegments: 9
1745: positiveSegments: 27, negativeSegments: 6
1747: positiveSegments: 0, negativeSegments: 9
1748: positiveSegments: 5, negativeSegments: 3
1749: positiveSegments: 0, negativeSegments: 6
1752: positiveSegments: 42, negativeSegments: 3
1753: positiveSegments: 6, negativeSegments: 0
1756: positiveSegments: 4, negativeSegments: 6
1757: positiveSegments: 0, negativeSegments: 6
1759: positiveSegments: 2, negativeSegments: 0
1761: positiveSegments: 26, negativeSegments: 0
1762: positiveSegments: 0, negativeSegments: 9
1763: positiveSegments: 5, negativeSegments: 0
1765: positiveSegments: 10, negativeSegments: 6
1766: positiveSegments: 16, negativeSegments: 0
1768: positiveSegments: 8, negativeSegments: 0
1771: positiveSegments: 9, negativeSegments: 0
1773: positiveSegments: 6, negativeSegments: 0
1775: positiveSegments: 11, negativeSegments: 0
1777: positiveSegments: 0, negativeSegments: 12
1779: positiveSegments: 2, negativeSegments: 12
1783: positiveSegments: 29, negativeSegments: 3
1784: positiveSegments: 12, negativeSegments: 0
1785: positiveSegments: 25, negativeSegments: 0
1793: positiveSegments: 13, negativeSegments: 0
1799: positiveSegments: 41, negativeSegments: 3
1800: positiveSegments: 0, negativeSegments: 3
1802: positiveSegments: 4, negativeSegments: 6
1803: positiveSegments: 62, negativeSegments: 21
1805: positiveSegments: 8, negativeSegments: 0
1809: positiveSegments: 3, negativeSegments: 12
1810: positiveSegments: 2, negativeSegments: 0
1812: positiveSegments: 15, negativeSegments: 0
1814: positiveSegments: 6, negativeSegments: 3
1816: positiveSegments: 18, negativeSegments: 15
1819: positiveSegments: 21, negativeSegments: 0
1820: positiveSegments: 17, negativeSegments: 0
1822: positiveSegments: 9, negativeSegments: 3
1823: positiveSegments: 4, negativeSegments: 6
1825: positiveSegments: 0, negativeSegments: 3
1826: positiveSegments: 6, negativeSegments: 0
1832: positiveSegments: 32, negativeSegments: 3
1833: positiveSegments: 30, negativeSegments: 0
1834: positiveSegments: 15, negativeSegments: 0
1835: positiveSegments: 23, negativeSegments: 0
1836: positiveSegments: 8, negativeSegments: 0
1837: positiveSegments: 2, negativeSegments: 3
1838: positiveSegments: 12, negativeSegments: 6
1840: positiveSegments: 5, negativeSegments: 3
count processed: 900, current case index: 1843
1843: positiveSegments: 17, negativeSegments: 6
1844: positiveSegments: 12, negativeSegments: 0
1846: positiveSegments: 5, negativeSegments: 0
1848: positiveSegments: 10, negativeSegments: 6
1852: positiveSegments: 22, negativeSegments: 3
1853: positiveSegments: 2, negativeSegments: 9
1854: positiveSegments: 16, negativeSegments: 0
1855: positiveSegments: 28, negativeSegments: 0
1862: positiveSegments: 19, negativeSegments: 9
1864: exit early, no segments to save
1864: positiveSegments: 0, negativeSegments: 0
1865: positiveSegments: 7, negativeSegments: 0
1866: positiveSegments: 4, negativeSegments: 0
1869: positiveSegments: 0, negativeSegments: 3
1872: positiveSegments: 15, negativeSegments: 0
1873: positiveSegments: 16, negativeSegments: 9
1874: positiveSegments: 8, negativeSegments: 0
1875: exit early, no segments to save
1875: positiveSegments: 0, negativeSegments: 0
1876: exit early, no segments to save
1876: positiveSegments: 0, negativeSegments: 0
1882: positiveSegments: 0, negativeSegments: 3
1884: positiveSegments: 17, negativeSegments: 12
1885: positiveSegments: 0, negativeSegments: 3
1886: positiveSegments: 10, negativeSegments: 6
1888: positiveSegments: 14, negativeSegments: 3
1891: positiveSegments: 0, negativeSegments: 6
1892: positiveSegments: 16, negativeSegments: 9
1893: positiveSegments: 20, negativeSegments: 12
1894: positiveSegments: 8, negativeSegments: 0
1896: positiveSegments: 6, negativeSegments: 3
1898: positiveSegments: 10, negativeSegments: 0
1899: positiveSegments: 0, negativeSegments: 6
1900: positiveSegments: 32, negativeSegments: 3
1901: positiveSegments: 17, negativeSegments: 6
1903: positiveSegments: 18, negativeSegments: 6
1907: positiveSegments: 4, negativeSegments: 0
1910: positiveSegments: 3, negativeSegments: 0
1912: positiveSegments: 10, negativeSegments: 21
1913: positiveSegments: 15, negativeSegments: 0
1914: positiveSegments: 11, negativeSegments: 3
1915: positiveSegments: 3, negativeSegments: 0
1916: positiveSegments: 0, negativeSegments: 15
1918: positiveSegments: 17, negativeSegments: 9
1920: positiveSegments: 12, negativeSegments: 3
1922: positiveSegments: 4, negativeSegments: 9
1925: positiveSegments: 0, negativeSegments: 3
1926: positiveSegments: 4, negativeSegments: 0
1928: positiveSegments: 28, negativeSegments: 0
1931: positiveSegments: 7, negativeSegments: 0
1932: positiveSegments: 29, negativeSegments: 6
1933: positiveSegments: 4, negativeSegments: 0
1934: positiveSegments: 26, negativeSegments: 0
1935: positiveSegments: 19, negativeSegments: 12
1936: positiveSegments: 65, negativeSegments: 3
1937: positiveSegments: 12, negativeSegments: 9
1938: positiveSegments: 12, negativeSegments: 15
1941: positiveSegments: 38, negativeSegments: 6
1942: positiveSegments: 6, negativeSegments: 0
1943: exit early, no segments to save
1943: positiveSegments: 0, negativeSegments: 0
1944: positiveSegments: 11, negativeSegments: 3
1947: positiveSegments: 4, negativeSegments: 6
1949: positiveSegments: 8, negativeSegments: 6
1950: positiveSegments: 0, negativeSegments: 6
1955: positiveSegments: 7, negativeSegments: 6
1956: positiveSegments: 7, negativeSegments: 3
1957: positiveSegments: 7, negativeSegments: 0
1959: positiveSegments: 19, negativeSegments: 3
1961: positiveSegments: 23, negativeSegments: 12
1963: positiveSegments: 21, negativeSegments: 9
1965: positiveSegments: 15, negativeSegments: 0
1966: positiveSegments: 8, negativeSegments: 0
1969: positiveSegments: 3, negativeSegments: 6
1973: positiveSegments: 1, negativeSegments: 9
1976: positiveSegments: 21, negativeSegments: 12
1978: positiveSegments: 16, negativeSegments: 6
1985: positiveSegments: 8, negativeSegments: 0
1986: positiveSegments: 2, negativeSegments: 0
1988: positiveSegments: 2, negativeSegments: 6
1993: positiveSegments: 1, negativeSegments: 6
1994: positiveSegments: 4, negativeSegments: 3
1995: positiveSegments: 12, negativeSegments: 0
1996: positiveSegments: 12, negativeSegments: 3
2000: positiveSegments: 0, negativeSegments: 6
2002: positiveSegments: 7, negativeSegments: 6
2004: positiveSegments: 6, negativeSegments: 6
2009: exit early, no segments to save
2009: positiveSegments: 0, negativeSegments: 0
2010: positiveSegments: 8, negativeSegments: 3
2011: positiveSegments: 4, negativeSegments: 18
2012: positiveSegments: 3, negativeSegments: 0
2014: positiveSegments: 7, negativeSegments: 9
2017: positiveSegments: 13, negativeSegments: 0
2018: positiveSegments: 22, negativeSegments: 3
2020: positiveSegments: 11, negativeSegments: 0
2025: positiveSegments: 6, negativeSegments: 0
2026: positiveSegments: 13, negativeSegments: 0
2028: positiveSegments: 1, negativeSegments: 21
2029: positiveSegments: 0, negativeSegments: 9
2034: exit early, no segments to save
2034: positiveSegments: 0, negativeSegments: 0
2040: positiveSegments: 10, negativeSegments: 6
2041: positiveSegments: 19, negativeSegments: 3
2044: positiveSegments: 8, negativeSegments: 6
2046: positiveSegments: 4, negativeSegments: 0
count processed: 1000, current case index: 2049
2049: positiveSegments: 4, negativeSegments: 3
2051: exit early, no segments to save
2051: positiveSegments: 0, negativeSegments: 0
2055: positiveSegments: 12, negativeSegments: 12
2057: positiveSegments: 16, negativeSegments: 0
2058: positiveSegments: 8, negativeSegments: 0
2060: positiveSegments: 4, negativeSegments: 15
2061: positiveSegments: 23, negativeSegments: 0
2062: positiveSegments: 0, negativeSegments: 12
2064: positiveSegments: 16, negativeSegments: 0
2066: positiveSegments: 11, negativeSegments: 6
2067: positiveSegments: 19, negativeSegments: 3
2068: positiveSegments: 14, negativeSegments: 0
2072: positiveSegments: 4, negativeSegments: 6
2074: exit early, no segments to save
2074: positiveSegments: 0, negativeSegments: 0
2075: positiveSegments: 0, negativeSegments: 3
2081: positiveSegments: 2, negativeSegments: 0
2082: positiveSegments: 11, negativeSegments: 9
2086: positiveSegments: 10, negativeSegments: 0
2088: positiveSegments: 0, negativeSegments: 6
2097: positiveSegments: 2, negativeSegments: 0
2098: positiveSegments: 0, negativeSegments: 3
2106: positiveSegments: 0, negativeSegments: 15
2111: positiveSegments: 2, negativeSegments: 0
2112: positiveSegments: 8, negativeSegments: 0
2114: positiveSegments: 4, negativeSegments: 3
2117: positiveSegments: 0, negativeSegments: 3
2118: positiveSegments: 0, negativeSegments: 12
2121: positiveSegments: 8, negativeSegments: 3
2130: positiveSegments: 0, negativeSegments: 9
2132: positiveSegments: 7, negativeSegments: 0
2133: positiveSegments: 9, negativeSegments: 0
2135: exit early, no segments to save
2135: positiveSegments: 0, negativeSegments: 0
2136: positiveSegments: 25, negativeSegments: 9
2139: positiveSegments: 0, negativeSegments: 6
2142: positiveSegments: 8, negativeSegments: 0
2147: positiveSegments: 17, negativeSegments: 9
2148: positiveSegments: 4, negativeSegments: 6
2149: positiveSegments: 4, negativeSegments: 6
2150: positiveSegments: 4, negativeSegments: 6
2153: positiveSegments: 41, negativeSegments: 12
2154: positiveSegments: 0, negativeSegments: 9
2157: exit early, no segments to save
2157: positiveSegments: 0, negativeSegments: 0
2158: positiveSegments: 0, negativeSegments: 3
2161: positiveSegments: 20, negativeSegments: 3
2162: nothing saved, all segments filtered
2162: positiveSegments: 2, negativeSegments: 0
2163: positiveSegments: 4, negativeSegments: 3
2165: positiveSegments: 22, negativeSegments: 12
2168: positiveSegments: 31, negativeSegments: 3
2169: positiveSegments: 2, negativeSegments: 9
2170: exit early, no segments to save
2170: positiveSegments: 0, negativeSegments: 0
2172: positiveSegments: 6, negativeSegments: 18
2174: exit early, no segments to save
2174: positiveSegments: 0, negativeSegments: 0
2175: positiveSegments: 0, negativeSegments: 3
2176: positiveSegments: 8, negativeSegments: 3
2178: exit early, no segments to save
2178: positiveSegments: 0, negativeSegments: 0
2183: positiveSegments: 6, negativeSegments: 9
2185: positiveSegments: 8, negativeSegments: 12
2187: positiveSegments: 7, negativeSegments: 0
2192: positiveSegments: 19, negativeSegments: 9
2194: positiveSegments: 3, negativeSegments: 18
2195: positiveSegments: 8, negativeSegments: 6
2196: positiveSegments: 4, negativeSegments: 3
2197: positiveSegments: 25, negativeSegments: 6
2201: positiveSegments: 21, negativeSegments: 0
2205: positiveSegments: 6, negativeSegments: 3
2206: positiveSegments: 8, negativeSegments: 3
2210: positiveSegments: 12, negativeSegments: 3
2213: positiveSegments: 0, negativeSegments: 3
2214: positiveSegments: 10, negativeSegments: 3
2218: exit early, no segments to save
2218: positiveSegments: 0, negativeSegments: 0
2219: nothing saved, all segments filtered
2219: positiveSegments: 13, negativeSegments: 0
2221: positiveSegments: 4, negativeSegments: 0
2222: positiveSegments: 6, negativeSegments: 0
2223: positiveSegments: 4, negativeSegments: 0
2224: positiveSegments: 5, negativeSegments: 0
2225: positiveSegments: 15, negativeSegments: 12
2229: positiveSegments: 0, negativeSegments: 9
2231: positiveSegments: 14, negativeSegments: 9
2236: positiveSegments: 17, negativeSegments: 3
2238: positiveSegments: 2, negativeSegments: 24
2241: positiveSegments: 0, negativeSegments: 9
2242: positiveSegments: 6, negativeSegments: 6
2243: positiveSegments: 20, negativeSegments: 3
2244: positiveSegments: 0, negativeSegments: 6
2246: positiveSegments: 8, negativeSegments: 3
2248: positiveSegments: 3, negativeSegments: 3
2249: positiveSegments: 4, negativeSegments: 0
2251: positiveSegments: 3, negativeSegments: 0
2252: positiveSegments: 14, negativeSegments: 9
2253: nothing saved, all segments filtered
2253: positiveSegments: 1, negativeSegments: 0
2255: positiveSegments: 0, negativeSegments: 3
2258: positiveSegments: 4, negativeSegments: 3
2261: positiveSegments: 18, negativeSegments: 0
2265: positiveSegments: 14, negativeSegments: 0
2267: positiveSegments: 18, negativeSegments: 3
2272: positiveSegments: 15, negativeSegments: 0
2273: positiveSegments: 0, negativeSegments: 12
2275: positiveSegments: 14, negativeSegments: 3
2279: positiveSegments: 8, negativeSegments: 0
2280: positiveSegments: 11, negativeSegments: 6
count processed: 1100, current case index: 2281
2281: exit early, no segments to save
2281: positiveSegments: 0, negativeSegments: 0
2282: positiveSegments: 0, negativeSegments: 3
2283: positiveSegments: 0, negativeSegments: 3
2284: positiveSegments: 7, negativeSegments: 3
2291: positiveSegments: 4, negativeSegments: 0
2295: positiveSegments: 5, negativeSegments: 0
2296: positiveSegments: 4, negativeSegments: 9
2298: positiveSegments: 4, negativeSegments: 3
2299: positiveSegments: 4, negativeSegments: 3
2300: positiveSegments: 8, negativeSegments: 21
2302: positiveSegments: 4, negativeSegments: 9
2304: positiveSegments: 13, negativeSegments: 6
2305: positiveSegments: 3, negativeSegments: 6
2306: positiveSegments: 11, negativeSegments: 6
2307: positiveSegments: 4, negativeSegments: 12
2309: positiveSegments: 11, negativeSegments: 0
2310: positiveSegments: 6, negativeSegments: 15
2311: positiveSegments: 14, negativeSegments: 0
2313: positiveSegments: 2, negativeSegments: 0
2315: positiveSegments: 9, negativeSegments: 0
2317: positiveSegments: 4, negativeSegments: 0
2318: positiveSegments: 31, negativeSegments: 0
2319: positiveSegments: 3, negativeSegments: 3
2321: positiveSegments: 4, negativeSegments: 3
2324: positiveSegments: 18, negativeSegments: 3
2325: positiveSegments: 17, negativeSegments: 3
2326: positiveSegments: 35, negativeSegments: 0
2327: positiveSegments: 11, negativeSegments: 0
2331: positiveSegments: 11, negativeSegments: 0
2332: positiveSegments: 34, negativeSegments: 6
2333: positiveSegments: 4, negativeSegments: 3
2334: positiveSegments: 21, negativeSegments: 3
2335: positiveSegments: 8, negativeSegments: 0
2336: positiveSegments: 30, negativeSegments: 15
2337: positiveSegments: 5, negativeSegments: 6
2339: positiveSegments: 11, negativeSegments: 0
2340: positiveSegments: 25, negativeSegments: 0
2341: positiveSegments: 3, negativeSegments: 3
2345: positiveSegments: 4, negativeSegments: 3
2346: positiveSegments: 0, negativeSegments: 3
2348: positiveSegments: 4, negativeSegments: 6
2349: positiveSegments: 16, negativeSegments: 6
2352: positiveSegments: 13, negativeSegments: 15
2353: positiveSegments: 9, negativeSegments: 15
2354: positiveSegments: 3, negativeSegments: 12
2356: positiveSegments: 14, negativeSegments: 0
2357: positiveSegments: 0, negativeSegments: 6
2359: positiveSegments: 18, negativeSegments: 0
2364: positiveSegments: 6, negativeSegments: 0
2365: positiveSegments: 4, negativeSegments: 0
2371: positiveSegments: 14, negativeSegments: 0
2372: positiveSegments: 0, negativeSegments: 6
2373: positiveSegments: 17, negativeSegments: 6
2375: positiveSegments: 16, negativeSegments: 15
2377: positiveSegments: 10, negativeSegments: 3
2379: positiveSegments: 4, negativeSegments: 0
2380: positiveSegments: 8, negativeSegments: 12
2382: positiveSegments: 5, negativeSegments: 0
2383: positiveSegments: 19, negativeSegments: 3
2389: positiveSegments: 15, negativeSegments: 9
2392: positiveSegments: 4, negativeSegments: 9
2393: positiveSegments: 12, negativeSegments: 3
2394: positiveSegments: 0, negativeSegments: 6
2396: positiveSegments: 17, negativeSegments: 0
2401: positiveSegments: 0, negativeSegments: 6
2405: positiveSegments: 16, negativeSegments: 3
2409: positiveSegments: 4, negativeSegments: 0
2411: positiveSegments: 14, negativeSegments: 0
2412: exit early, no segments to save
2412: positiveSegments: 0, negativeSegments: 0
2413: positiveSegments: 4, negativeSegments: 0
2416: positiveSegments: 7, negativeSegments: 9
2417: positiveSegments: 4, negativeSegments: 3
2419: positiveSegments: 4, negativeSegments: 0
2420: positiveSegments: 0, negativeSegments: 21
2421: positiveSegments: 0, negativeSegments: 9
2422: positiveSegments: 11, negativeSegments: 0
2424: positiveSegments: 23, negativeSegments: 3
2425: positiveSegments: 4, negativeSegments: 6
2427: positiveSegments: 2, negativeSegments: 9
2428: positiveSegments: 7, negativeSegments: 3
2432: positiveSegments: 11, negativeSegments: 0
2433: positiveSegments: 38, negativeSegments: 21
2434: exit early, no segments to save
2434: positiveSegments: 0, negativeSegments: 0
2435: positiveSegments: 4, negativeSegments: 3
2436: positiveSegments: 0, negativeSegments: 3
2438: positiveSegments: 0, negativeSegments: 6
2441: positiveSegments: 23, negativeSegments: 6
2442: positiveSegments: 6, negativeSegments: 0
2443: positiveSegments: 7, negativeSegments: 30
2444: positiveSegments: 6, negativeSegments: 0
2445: positiveSegments: 12, negativeSegments: 3
2447: positiveSegments: 0, negativeSegments: 3
2450: exit early, no segments to save
2450: positiveSegments: 0, negativeSegments: 0
2452: positiveSegments: 17, negativeSegments: 0
2453: positiveSegments: 30, negativeSegments: 6
2455: positiveSegments: 6, negativeSegments: 6
2458: positiveSegments: 25, negativeSegments: 9
2460: positiveSegments: 3, negativeSegments: 0
2462: positiveSegments: 17, negativeSegments: 3
2466: positiveSegments: 2, negativeSegments: 0
count processed: 1200, current case index: 2469
2469: positiveSegments: 4, negativeSegments: 6
2470: positiveSegments: 24, negativeSegments: 0
2471: positiveSegments: 10, negativeSegments: 0
2472: positiveSegments: 8, negativeSegments: 6
2473: exit early, no segments to save
2473: positiveSegments: 0, negativeSegments: 0
2474: positiveSegments: 2, negativeSegments: 6
2479: exit early, no segments to save
2479: positiveSegments: 0, negativeSegments: 0
2480: positiveSegments: 4, negativeSegments: 9
2481: positiveSegments: 0, negativeSegments: 3
2482: positiveSegments: 0, negativeSegments: 3
2483: positiveSegments: 4, negativeSegments: 0
2485: positiveSegments: 0, negativeSegments: 6
2487: positiveSegments: 35, negativeSegments: 9
2489: positiveSegments: 38, negativeSegments: 6
2493: positiveSegments: 26, negativeSegments: 3
2494: positiveSegments: 19, negativeSegments: 0
2495: positiveSegments: 4, negativeSegments: 9
2496: exit early, no segments to save
2496: positiveSegments: 0, negativeSegments: 0
2497: positiveSegments: 12, negativeSegments: 9
2500: positiveSegments: 4, negativeSegments: 0
2501: positiveSegments: 0, negativeSegments: 18
2503: positiveSegments: 0, negativeSegments: 3
2507: positiveSegments: 6, negativeSegments: 6
2508: positiveSegments: 0, negativeSegments: 9
2509: exit early, no segments to save
2509: positiveSegments: 0, negativeSegments: 0
2510: positiveSegments: 7, negativeSegments: 0
2511: positiveSegments: 5, negativeSegments: 0
2516: exit early, no segments to save
2516: positiveSegments: 0, negativeSegments: 0
2517: positiveSegments: 10, negativeSegments: 0
2519: positiveSegments: 25, negativeSegments: 3
2521: positiveSegments: 14, negativeSegments: 0
2523: positiveSegments: 8, negativeSegments: 3
2527: positiveSegments: 14, negativeSegments: 6
2528: exit early, no segments to save
2528: positiveSegments: 0, negativeSegments: 0
2532: positiveSegments: 4, negativeSegments: 0
2533: positiveSegments: 5, negativeSegments: 0
2535: positiveSegments: 4, negativeSegments: 0
2537: positiveSegments: 8, negativeSegments: 6
2539: positiveSegments: 3, negativeSegments: 0
2542: positiveSegments: 0, negativeSegments: 3
2544: positiveSegments: 10, negativeSegments: 12
2547: positiveSegments: 16, negativeSegments: 6
2549: exit early, no segments to save
2549: positiveSegments: 0, negativeSegments: 0
2553: positiveSegments: 27, negativeSegments: 0
2558: positiveSegments: 4, negativeSegments: 3
2559: positiveSegments: 14, negativeSegments: 0
2561: exit early, no segments to save
2561: positiveSegments: 0, negativeSegments: 0
2562: positiveSegments: 0, negativeSegments: 6
2566: positiveSegments: 0, negativeSegments: 3
2568: positiveSegments: 1, negativeSegments: 12
2569: exit early, no segments to save
2569: positiveSegments: 0, negativeSegments: 0
2572: exit early, no segments to save
2572: positiveSegments: 0, negativeSegments: 0
2575: exit early, no segments to save
2575: positiveSegments: 0, negativeSegments: 0
2576: exit early, no segments to save
2576: positiveSegments: 0, negativeSegments: 0
2578: positiveSegments: 4, negativeSegments: 6
2580: positiveSegments: 21, negativeSegments: 6
2583: positiveSegments: 6, negativeSegments: 0
2584: positiveSegments: 17, negativeSegments: 9
2585: positiveSegments: 18, negativeSegments: 0
2587: positiveSegments: 14, negativeSegments: 0
2589: positiveSegments: 0, negativeSegments: 6
2590: positiveSegments: 1, negativeSegments: 6
2593: exit early, no segments to save
2593: positiveSegments: 0, negativeSegments: 0
2594: positiveSegments: 10, negativeSegments: 3
2596: positiveSegments: 6, negativeSegments: 3
2597: positiveSegments: 2, negativeSegments: 0
2601: positiveSegments: 6, negativeSegments: 6
2605: positiveSegments: 1, negativeSegments: 0
2606: positiveSegments: 8, negativeSegments: 9
2607: exit early, no segments to save
2607: positiveSegments: 0, negativeSegments: 0
2608: positiveSegments: 10, negativeSegments: 0
2609: positiveSegments: 4, negativeSegments: 3
2611: positiveSegments: 18, negativeSegments: 3
2612: positiveSegments: 10, negativeSegments: 0
2613: positiveSegments: 0, negativeSegments: 3
2616: exit early, no segments to save
2616: positiveSegments: 0, negativeSegments: 0
2618: positiveSegments: 7, negativeSegments: 15
2619: positiveSegments: 11, negativeSegments: 3
2622: positiveSegments: 17, negativeSegments: 3
2624: positiveSegments: 8, negativeSegments: 3
2627: positiveSegments: 8, negativeSegments: 0
2628: positiveSegments: 2, negativeSegments: 9
2630: positiveSegments: 6, negativeSegments: 6
2631: positiveSegments: 12, negativeSegments: 0
2632: positiveSegments: 15, negativeSegments: 0
2637: positiveSegments: 2, negativeSegments: 3
2639: positiveSegments: 4, negativeSegments: 0
2641: positiveSegments: 0, negativeSegments: 6
2644: positiveSegments: 9, negativeSegments: 0
2646: exit early, no segments to save
2646: positiveSegments: 0, negativeSegments: 0
2648: positiveSegments: 12, negativeSegments: 9
2652: positiveSegments: 0, negativeSegments: 15
2654: positiveSegments: 0, negativeSegments: 3
2655: positiveSegments: 3, negativeSegments: 6
2656: positiveSegments: 0, negativeSegments: 12
2657: positiveSegments: 0, negativeSegments: 9
2658: positiveSegments: 13, negativeSegments: 6
2662: positiveSegments: 11, negativeSegments: 21
2663: positiveSegments: 4, negativeSegments: 6
2664: positiveSegments: 8, negativeSegments: 15
count processed: 1300, current case index: 2665
2665: positiveSegments: 0, negativeSegments: 3
2667: positiveSegments: 17, negativeSegments: 9
2670: exit early, no segments to save
2670: positiveSegments: 0, negativeSegments: 0
2671: positiveSegments: 13, negativeSegments: 3
2673: exit early, no segments to save
2673: positiveSegments: 0, negativeSegments: 0
2674: positiveSegments: 8, negativeSegments: 9
2676: positiveSegments: 4, negativeSegments: 0
2678: positiveSegments: 4, negativeSegments: 3
2680: positiveSegments: 17, negativeSegments: 0
2687: positiveSegments: 9, negativeSegments: 0
2688: positiveSegments: 23, negativeSegments: 3
2690: positiveSegments: 8, negativeSegments: 3
2693: positiveSegments: 19, negativeSegments: 6
2695: exit early, no segments to save
2695: positiveSegments: 0, negativeSegments: 0
2697: positiveSegments: 0, negativeSegments: 9
2698: positiveSegments: 4, negativeSegments: 0
2699: positiveSegments: 5, negativeSegments: 9
2700: positiveSegments: 7, negativeSegments: 0
2701: positiveSegments: 0, negativeSegments: 6
2703: positiveSegments: 4, negativeSegments: 15
2705: positiveSegments: 2, negativeSegments: 9
2706: positiveSegments: 36, negativeSegments: 3
2712: positiveSegments: 13, negativeSegments: 3
2713: positiveSegments: 0, negativeSegments: 3
2716: positiveSegments: 0, negativeSegments: 6
2717: positiveSegments: 8, negativeSegments: 0
2722: positiveSegments: 50, negativeSegments: 0
2724: positiveSegments: 10, negativeSegments: 12
2732: exit early, no segments to save
2732: positiveSegments: 0, negativeSegments: 0
2735: positiveSegments: 4, negativeSegments: 3
2736: exit early, no segments to save
2736: positiveSegments: 0, negativeSegments: 0
2738: positiveSegments: 9, negativeSegments: 3
2741: positiveSegments: 18, negativeSegments: 6
2742: positiveSegments: 19, negativeSegments: 3
2744: positiveSegments: 4, negativeSegments: 6
2746: positiveSegments: 4, negativeSegments: 0
2747: positiveSegments: 7, negativeSegments: 9
2749: positiveSegments: 10, negativeSegments: 3
2750: positiveSegments: 4, negativeSegments: 6
2751: positiveSegments: 25, negativeSegments: 6
2755: positiveSegments: 12, negativeSegments: 9
2760: exit early, no segments to save
2760: positiveSegments: 0, negativeSegments: 0
2761: positiveSegments: 10, negativeSegments: 3
2762: positiveSegments: 0, negativeSegments: 3
2763: positiveSegments: 4, negativeSegments: 0
2764: positiveSegments: 26, negativeSegments: 6
2765: positiveSegments: 14, negativeSegments: 0
2766: positiveSegments: 0, negativeSegments: 12
2769: positiveSegments: 4, negativeSegments: 9
2771: exit early, no segments to save
2771: positiveSegments: 0, negativeSegments: 0
2774: positiveSegments: 5, negativeSegments: 0
2775: positiveSegments: 18, negativeSegments: 3
2777: positiveSegments: 4, negativeSegments: 3
2778: positiveSegments: 16, negativeSegments: 6
2779: positiveSegments: 4, negativeSegments: 0
2781: positiveSegments: 21, negativeSegments: 9
2783: positiveSegments: 18, negativeSegments: 0
2785: positiveSegments: 25, negativeSegments: 3
2788: positiveSegments: 3, negativeSegments: 0
2795: positiveSegments: 9, negativeSegments: 6
2799: positiveSegments: 4, negativeSegments: 0
2800: positiveSegments: 5, negativeSegments: 12
2803: positiveSegments: 2, negativeSegments: 0
2804: positiveSegments: 12, negativeSegments: 9
2806: positiveSegments: 3, negativeSegments: 0
2807: positiveSegments: 5, negativeSegments: 6
2809: positiveSegments: 11, negativeSegments: 3
2814: positiveSegments: 10, negativeSegments: 6
2815: positiveSegments: 19, negativeSegments: 0
2819: positiveSegments: 17, negativeSegments: 0
2820: positiveSegments: 5, negativeSegments: 9
2823: positiveSegments: 13, negativeSegments: 3
2824: positiveSegments: 22, negativeSegments: 9
2827: positiveSegments: 17, negativeSegments: 0
2828: positiveSegments: 26, negativeSegments: 6
2829: positiveSegments: 15, negativeSegments: 3
2830: positiveSegments: 8, negativeSegments: 0
2835: positiveSegments: 30, negativeSegments: 0
2836: positiveSegments: 0, negativeSegments: 9
2837: positiveSegments: 11, negativeSegments: 6
2839: positiveSegments: 5, negativeSegments: 3
2845: exit early, no segments to save
2845: positiveSegments: 0, negativeSegments: 0
2847: positiveSegments: 4, negativeSegments: 12
2848: positiveSegments: 6, negativeSegments: 12
2849: positiveSegments: 15, negativeSegments: 0
2850: positiveSegments: 6, negativeSegments: 6
2851: positiveSegments: 23, negativeSegments: 3
2854: positiveSegments: 4, negativeSegments: 3
2858: positiveSegments: 25, negativeSegments: 3
2859: positiveSegments: 5, negativeSegments: 0
2860: positiveSegments: 8, negativeSegments: 18
2861: positiveSegments: 10, negativeSegments: 0
2863: positiveSegments: 4, negativeSegments: 6
2864: positiveSegments: 8, negativeSegments: 3
2868: positiveSegments: 6, negativeSegments: 0
2871: positiveSegments: 3, negativeSegments: 3
2872: positiveSegments: 20, negativeSegments: 6
2876: positiveSegments: 13, negativeSegments: 9
2877: nothing saved, all segments filtered
2877: positiveSegments: 2, negativeSegments: 0
2883: positiveSegments: 8, negativeSegments: 3
count processed: 1400, current case index: 2888
2888: positiveSegments: 2, negativeSegments: 9
2889: positiveSegments: 12, negativeSegments: 0
2890: positiveSegments: 6, negativeSegments: 0
2891: positiveSegments: 11, negativeSegments: 0
2895: exit early, no segments to save
2895: positiveSegments: 0, negativeSegments: 0
2900: positiveSegments: 5, negativeSegments: 0
2903: positiveSegments: 6, negativeSegments: 0
2905: positiveSegments: 8, negativeSegments: 9
2906: exit early, no segments to save
2906: positiveSegments: 0, negativeSegments: 0
2909: exit early, no segments to save
2909: positiveSegments: 0, negativeSegments: 0
2910: positiveSegments: 0, negativeSegments: 3
2911: positiveSegments: 25, negativeSegments: 12
2914: positiveSegments: 0, negativeSegments: 6
2919: positiveSegments: 4, negativeSegments: 0
2922: positiveSegments: 4, negativeSegments: 0
2924: positiveSegments: 12, negativeSegments: 0
2927: positiveSegments: 0, negativeSegments: 3
2929: positiveSegments: 10, negativeSegments: 0
2930: positiveSegments: 1, negativeSegments: 6
2931: positiveSegments: 4, negativeSegments: 3
2935: positiveSegments: 17, negativeSegments: 0
2939: positiveSegments: 0, negativeSegments: 12
2940: positiveSegments: 18, negativeSegments: 6
2943: positiveSegments: 4, negativeSegments: 3
2944: positiveSegments: 4, negativeSegments: 9
2945: positiveSegments: 34, negativeSegments: 9
2947: positiveSegments: 20, negativeSegments: 0
2949: positiveSegments: 7, negativeSegments: 0
2952: positiveSegments: 4, negativeSegments: 21
2954: positiveSegments: 0, negativeSegments: 15
2955: positiveSegments: 4, negativeSegments: 3
2956: positiveSegments: 0, negativeSegments: 3
2958: positiveSegments: 0, negativeSegments: 3
2959: positiveSegments: 9, negativeSegments: 12
2960: positiveSegments: 25, negativeSegments: 0
2961: positiveSegments: 22, negativeSegments: 9
2964: positiveSegments: 13, negativeSegments: 0
2966: positiveSegments: 7, negativeSegments: 3
2970: positiveSegments: 3, negativeSegments: 12
2971: positiveSegments: 0, negativeSegments: 12
2972: positiveSegments: 9, negativeSegments: 0
2974: positiveSegments: 4, negativeSegments: 6
2975: positiveSegments: 4, negativeSegments: 9
2977: positiveSegments: 9, negativeSegments: 0
2980: positiveSegments: 8, negativeSegments: 3
2981: positiveSegments: 0, negativeSegments: 9
2982: positiveSegments: 13, negativeSegments: 9
2987: positiveSegments: 0, negativeSegments: 3
2991: positiveSegments: 17, negativeSegments: 6
2992: positiveSegments: 19, negativeSegments: 3
2993: positiveSegments: 17, negativeSegments: 3
2998: positiveSegments: 0, negativeSegments: 3
2999: positiveSegments: 8, negativeSegments: 0
3000: positiveSegments: 14, negativeSegments: 0
3001: positiveSegments: 11, negativeSegments: 3
3003: positiveSegments: 8, negativeSegments: 0
3004: positiveSegments: 0, negativeSegments: 9
3006: positiveSegments: 17, negativeSegments: 0
3009: positiveSegments: 26, negativeSegments: 3
3014: positiveSegments: 12, negativeSegments: 3
3015: nothing saved, all segments filtered
3015: positiveSegments: 1, negativeSegments: 0
3019: positiveSegments: 29, negativeSegments: 0
3020: positiveSegments: 5, negativeSegments: 0
3023: positiveSegments: 10, negativeSegments: 3
3024: positiveSegments: 29, negativeSegments: 3
3027: positiveSegments: 4, negativeSegments: 0
3028: positiveSegments: 16, negativeSegments: 3
3030: positiveSegments: 6, negativeSegments: 6
3034: positiveSegments: 13, negativeSegments: 0
3035: positiveSegments: 8, negativeSegments: 6
3037: positiveSegments: 21, negativeSegments: 12
3039: positiveSegments: 13, negativeSegments: 0
3040: positiveSegments: 23, negativeSegments: 6
3042: positiveSegments: 8, negativeSegments: 3
3044: positiveSegments: 20, negativeSegments: 3
3045: positiveSegments: 0, negativeSegments: 3
3046: exit early, no segments to save
3046: positiveSegments: 0, negativeSegments: 0
3047: positiveSegments: 12, negativeSegments: 0
3048: positiveSegments: 0, negativeSegments: 9
3050: positiveSegments: 6, negativeSegments: 0
3051: positiveSegments: 35, negativeSegments: 9
3057: positiveSegments: 5, negativeSegments: 3
3058: positiveSegments: 36, negativeSegments: 0
3059: positiveSegments: 4, negativeSegments: 9
3060: positiveSegments: 11, negativeSegments: 6
3065: positiveSegments: 5, negativeSegments: 0
3070: positiveSegments: 0, negativeSegments: 3
3072: exit early, no segments to save
3072: positiveSegments: 0, negativeSegments: 0
3073: positiveSegments: 4, negativeSegments: 3
3074: positiveSegments: 32, negativeSegments: 6
3075: positiveSegments: 8, negativeSegments: 3
3078: exit early, no segments to save
3078: positiveSegments: 0, negativeSegments: 0
3079: positiveSegments: 12, negativeSegments: 6
3080: positiveSegments: 2, negativeSegments: 0
3082: positiveSegments: 10, negativeSegments: 6
3085: positiveSegments: 6, negativeSegments: 0
3087: positiveSegments: 14, negativeSegments: 0
3088: positiveSegments: 5, negativeSegments: 0
3090: positiveSegments: 5, negativeSegments: 6
3091: positiveSegments: 8, negativeSegments: 0
count processed: 1500, current case index: 3092
3092: positiveSegments: 2, negativeSegments: 6
3093: positiveSegments: 14, negativeSegments: 12
3094: positiveSegments: 8, negativeSegments: 6
3095: positiveSegments: 12, negativeSegments: 0
3096: positiveSegments: 15, negativeSegments: 3
3097: positiveSegments: 12, negativeSegments: 0
3098: positiveSegments: 4, negativeSegments: 9
3101: positiveSegments: 15, negativeSegments: 0
3102: positiveSegments: 1, negativeSegments: 3
3103: positiveSegments: 0, negativeSegments: 3
3106: positiveSegments: 0, negativeSegments: 6
3107: positiveSegments: 12, negativeSegments: 3
3108: positiveSegments: 15, negativeSegments: 9
3110: positiveSegments: 0, negativeSegments: 9
3112: positiveSegments: 4, negativeSegments: 3
3113: positiveSegments: 24, negativeSegments: 6
3114: exit early, no segments to save
3114: positiveSegments: 0, negativeSegments: 0
3116: positiveSegments: 8, negativeSegments: 0
3118: positiveSegments: 0, negativeSegments: 12
3120: positiveSegments: 11, negativeSegments: 0
3121: positiveSegments: 0, negativeSegments: 12
3122: positiveSegments: 5, negativeSegments: 3
3125: positiveSegments: 0, negativeSegments: 6
3126: positiveSegments: 8, negativeSegments: 0
3128: exit early, no segments to save
3128: positiveSegments: 0, negativeSegments: 0
3130: positiveSegments: 0, negativeSegments: 9
3133: positiveSegments: 24, negativeSegments: 3
3134: positiveSegments: 15, negativeSegments: 21
3135: positiveSegments: 0, negativeSegments: 3
3136: positiveSegments: 12, negativeSegments: 0
3137: positiveSegments: 4, negativeSegments: 12
3138: positiveSegments: 18, negativeSegments: 0
3141: positiveSegments: 4, negativeSegments: 3
3142: positiveSegments: 15, negativeSegments: 0
3145: positiveSegments: 20, negativeSegments: 9
3146: positiveSegments: 32, negativeSegments: 3
3147: positiveSegments: 13, negativeSegments: 6
3148: positiveSegments: 29, negativeSegments: 12
3149: positiveSegments: 4, negativeSegments: 12
3150: positiveSegments: 4, negativeSegments: 9
3153: positiveSegments: 7, negativeSegments: 0
3154: positiveSegments: 22, negativeSegments: 6
3155: positiveSegments: 12, negativeSegments: 9
3157: positiveSegments: 16, negativeSegments: 0
3159: positiveSegments: 8, negativeSegments: 6
3161: positiveSegments: 16, negativeSegments: 3
3164: positiveSegments: 4, negativeSegments: 9
3165: positiveSegments: 26, negativeSegments: 0
3167: positiveSegments: 8, negativeSegments: 12
3169: positiveSegments: 12, negativeSegments: 0
3170: exit early, no segments to save
3170: positiveSegments: 0, negativeSegments: 0
3172: positiveSegments: 0, negativeSegments: 3
3173: positiveSegments: 23, negativeSegments: 3
3175: positiveSegments: 0, negativeSegments: 6
3176: exit early, no segments to save
3176: positiveSegments: 0, negativeSegments: 0
3180: positiveSegments: 8, negativeSegments: 0
3181: positiveSegments: 24, negativeSegments: 12
3184: positiveSegments: 4, negativeSegments: 6
3186: positiveSegments: 10, negativeSegments: 9
3188: positiveSegments: 33, negativeSegments: 0
3189: positiveSegments: 0, negativeSegments: 9
3193: positiveSegments: 20, negativeSegments: 24
3194: positiveSegments: 0, negativeSegments: 18
3196: positiveSegments: 0, negativeSegments: 9
3198: positiveSegments: 5, negativeSegments: 0
3200: positiveSegments: 4, negativeSegments: 0
3202: positiveSegments: 16, negativeSegments: 0
3203: positiveSegments: 4, negativeSegments: 12
3205: positiveSegments: 2, negativeSegments: 0
3207: exit early, no segments to save
3207: positiveSegments: 0, negativeSegments: 0
3208: positiveSegments: 0, negativeSegments: 3
3211: positiveSegments: 0, negativeSegments: 6
3216: positiveSegments: 9, negativeSegments: 3
3218: positiveSegments: 4, negativeSegments: 3
3221: positiveSegments: 3, negativeSegments: 3
3222: positiveSegments: 26, negativeSegments: 0
3223: positiveSegments: 0, negativeSegments: 12
3226: positiveSegments: 7, negativeSegments: 0
3228: positiveSegments: 35, negativeSegments: 6
3229: positiveSegments: 0, negativeSegments: 9
3231: positiveSegments: 12, negativeSegments: 6
3232: positiveSegments: 6, negativeSegments: 9
3233: positiveSegments: 22, negativeSegments: 3
3235: positiveSegments: 9, negativeSegments: 0
3240: positiveSegments: 4, negativeSegments: 6
3243: positiveSegments: 0, negativeSegments: 6
3247: positiveSegments: 33, negativeSegments: 6
3248: positiveSegments: 9, negativeSegments: 0
3249: positiveSegments: 2, negativeSegments: 0
3250: positiveSegments: 12, negativeSegments: 6
3255: positiveSegments: 27, negativeSegments: 0
3256: positiveSegments: 4, negativeSegments: 6
3260: positiveSegments: 6, negativeSegments: 3
3263: positiveSegments: 4, negativeSegments: 6
3267: positiveSegments: 0, negativeSegments: 18
3270: positiveSegments: 48, negativeSegments: 0
3271: positiveSegments: 15, negativeSegments: 6
3273: positiveSegments: 40, negativeSegments: 0
3275: positiveSegments: 9, negativeSegments: 3
3276: positiveSegments: 10, negativeSegments: 0
count processed: 1600, current case index: 3279
3279: positiveSegments: 23, negativeSegments: 3
3284: positiveSegments: 20, negativeSegments: 6
3286: positiveSegments: 21, negativeSegments: 0
3287: positiveSegments: 12, negativeSegments: 12
3291: positiveSegments: 26, negativeSegments: 6
3293: positiveSegments: 30, negativeSegments: 6
3295: positiveSegments: 8, negativeSegments: 0
3296: positiveSegments: 0, negativeSegments: 3
3297: positiveSegments: 18, negativeSegments: 3
3299: exit early, no segments to save
3299: positiveSegments: 0, negativeSegments: 0
3304: positiveSegments: 8, negativeSegments: 3
3307: positiveSegments: 13, negativeSegments: 6
3309: positiveSegments: 8, negativeSegments: 6
3310: positiveSegments: 26, negativeSegments: 0
3311: positiveSegments: 26, negativeSegments: 12
3312: positiveSegments: 6, negativeSegments: 3
3315: positiveSegments: 0, negativeSegments: 9
3317: positiveSegments: 8, negativeSegments: 6
3318: positiveSegments: 12, negativeSegments: 9
3321: positiveSegments: 4, negativeSegments: 6
3325: positiveSegments: 12, negativeSegments: 6
3327: positiveSegments: 4, negativeSegments: 3
3328: positiveSegments: 17, negativeSegments: 3
3329: positiveSegments: 9, negativeSegments: 0
3330: positiveSegments: 7, negativeSegments: 12
3331: positiveSegments: 19, negativeSegments: 3
3333: positiveSegments: 8, negativeSegments: 3
3334: positiveSegments: 33, negativeSegments: 9
3336: positiveSegments: 12, negativeSegments: 0
3338: positiveSegments: 14, negativeSegments: 3
3340: positiveSegments: 14, negativeSegments: 0
3342: positiveSegments: 8, negativeSegments: 9
3344: positiveSegments: 4, negativeSegments: 3
3345: positiveSegments: 24, negativeSegments: 3
3348: positiveSegments: 21, negativeSegments: 9
3352: positiveSegments: 18, negativeSegments: 6
3355: positiveSegments: 4, negativeSegments: 0
3357: positiveSegments: 4, negativeSegments: 0
3358: positiveSegments: 9, negativeSegments: 12
3359: positiveSegments: 10, negativeSegments: 0
3361: positiveSegments: 1, negativeSegments: 21
3362: exit early, no segments to save
3362: positiveSegments: 0, negativeSegments: 0
3363: positiveSegments: 4, negativeSegments: 12
3364: positiveSegments: 17, negativeSegments: 6
3365: exit early, no segments to save
3365: positiveSegments: 0, negativeSegments: 0
3366: exit early, no segments to save
3366: positiveSegments: 0, negativeSegments: 0
3367: positiveSegments: 3, negativeSegments: 3
3369: positiveSegments: 0, negativeSegments: 3
3373: positiveSegments: 30, negativeSegments: 9
3374: positiveSegments: 3, negativeSegments: 3
3375: positiveSegments: 1, negativeSegments: 3
3376: positiveSegments: 4, negativeSegments: 0
3377: exit early, no segments to save
3377: positiveSegments: 0, negativeSegments: 0
3379: positiveSegments: 0, negativeSegments: 3
3380: positiveSegments: 16, negativeSegments: 9
3381: positiveSegments: 0, negativeSegments: 15
3382: positiveSegments: 16, negativeSegments: 3
3383: positiveSegments: 8, negativeSegments: 3
3385: positiveSegments: 4, negativeSegments: 9
3389: positiveSegments: 5, negativeSegments: 3
3391: exit early, no segments to save
3391: positiveSegments: 0, negativeSegments: 0
3394: positiveSegments: 12, negativeSegments: 12
3396: positiveSegments: 3, negativeSegments: 0
3398: positiveSegments: 6, negativeSegments: 3
3399: positiveSegments: 4, negativeSegments: 3
3401: positiveSegments: 12, negativeSegments: 3
3404: positiveSegments: 18, negativeSegments: 3
3407: positiveSegments: 0, negativeSegments: 3
3409: positiveSegments: 14, negativeSegments: 3
3412: positiveSegments: 3, negativeSegments: 0
3413: exit early, no segments to save
3413: positiveSegments: 0, negativeSegments: 0
3414: exit early, no segments to save
3414: positiveSegments: 0, negativeSegments: 0
3415: positiveSegments: 28, negativeSegments: 6
3418: positiveSegments: 4, negativeSegments: 6
3422: positiveSegments: 4, negativeSegments: 3
3426: positiveSegments: 6, negativeSegments: 0
3427: positiveSegments: 4, negativeSegments: 0
3428: positiveSegments: 18, negativeSegments: 3
3429: positiveSegments: 15, negativeSegments: 6
3431: positiveSegments: 0, negativeSegments: 6
3434: positiveSegments: 0, negativeSegments: 3
3435: positiveSegments: 0, negativeSegments: 3
3436: positiveSegments: 4, negativeSegments: 6
3440: positiveSegments: 2, negativeSegments: 6
3441: positiveSegments: 0, negativeSegments: 3
3442: positiveSegments: 20, negativeSegments: 3
3447: positiveSegments: 11, negativeSegments: 3
3449: positiveSegments: 12, negativeSegments: 3
3450: positiveSegments: 8, negativeSegments: 6
3451: positiveSegments: 8, negativeSegments: 9
3453: positiveSegments: 0, negativeSegments: 6
3457: positiveSegments: 14, negativeSegments: 6
3458: positiveSegments: 8, negativeSegments: 3
3460: positiveSegments: 10, negativeSegments: 0
3462: positiveSegments: 9, negativeSegments: 6
3463: positiveSegments: 4, negativeSegments: 9
3464: positiveSegments: 8, negativeSegments: 6
3468: positiveSegments: 4, negativeSegments: 12
3470: positiveSegments: 12, negativeSegments: 0
3472: positiveSegments: 6, negativeSegments: 0
count processed: 1700, current case index: 3475
3475: positiveSegments: 4, negativeSegments: 3
3476: exit early, no segments to save
3476: positiveSegments: 0, negativeSegments: 0
3478: positiveSegments: 12, negativeSegments: 3
3479: positiveSegments: 0, negativeSegments: 3
3481: positiveSegments: 16, negativeSegments: 0
3483: positiveSegments: 24, negativeSegments: 15
3488: positiveSegments: 4, negativeSegments: 6
3492: positiveSegments: 10, negativeSegments: 0
3499: positiveSegments: 0, negativeSegments: 15
3500: exit early, no segments to save
3500: positiveSegments: 0, negativeSegments: 0
3501: positiveSegments: 4, negativeSegments: 3
3502: positiveSegments: 8, negativeSegments: 9
3503: positiveSegments: 5, negativeSegments: 9
3505: positiveSegments: 4, negativeSegments: 6
3506: positiveSegments: 17, negativeSegments: 0
3513: positiveSegments: 0, negativeSegments: 15
3514: positiveSegments: 12, negativeSegments: 0
3515: positiveSegments: 8, negativeSegments: 6
3516: positiveSegments: 12, negativeSegments: 9
3517: positiveSegments: 7, negativeSegments: 0
3518: exit early, no segments to save
3518: positiveSegments: 0, negativeSegments: 0
3521: positiveSegments: 38, negativeSegments: 6
3524: positiveSegments: 37, negativeSegments: 3
3526: positiveSegments: 3, negativeSegments: 3
3527: positiveSegments: 4, negativeSegments: 6
3528: positiveSegments: 0, negativeSegments: 3
3529: positiveSegments: 6, negativeSegments: 0
3532: positiveSegments: 2, negativeSegments: 3
3533: exit early, no segments to save
3533: positiveSegments: 0, negativeSegments: 0
3535: positiveSegments: 14, negativeSegments: 0
3537: positiveSegments: 4, negativeSegments: 9
3538: exit early, no segments to save
3538: positiveSegments: 0, negativeSegments: 0
3544: positiveSegments: 0, negativeSegments: 6
3545: positiveSegments: 7, negativeSegments: 0
3546: positiveSegments: 23, negativeSegments: 6
3549: positiveSegments: 27, negativeSegments: 0
3550: positiveSegments: 22, negativeSegments: 3
3555: positiveSegments: 0, negativeSegments: 6
3558: positiveSegments: 4, negativeSegments: 12
3559: positiveSegments: 25, negativeSegments: 9
3560: positiveSegments: 3, negativeSegments: 6
3562: positiveSegments: 1, negativeSegments: 3
3564: positiveSegments: 2, negativeSegments: 3
3565: positiveSegments: 12, negativeSegments: 6
3566: positiveSegments: 30, negativeSegments: 3
3567: positiveSegments: 0, negativeSegments: 6
3568: positiveSegments: 11, negativeSegments: 3
3569: positiveSegments: 10, negativeSegments: 0
3570: positiveSegments: 4, negativeSegments: 0
3571: positiveSegments: 8, negativeSegments: 15
3572: positiveSegments: 36, negativeSegments: 6
3573: positiveSegments: 8, negativeSegments: 3
3576: positiveSegments: 9, negativeSegments: 6
3581: positiveSegments: 4, negativeSegments: 3
3582: positiveSegments: 11, negativeSegments: 9
3585: positiveSegments: 9, negativeSegments: 0
3588: positiveSegments: 34, negativeSegments: 0
3589: positiveSegments: 0, negativeSegments: 3
3593: positiveSegments: 4, negativeSegments: 9
3594: positiveSegments: 4, negativeSegments: 18
3596: positiveSegments: 0, negativeSegments: 3
3602: positiveSegments: 7, negativeSegments: 6
3603: positiveSegments: 8, negativeSegments: 3
3606: positiveSegments: 0, negativeSegments: 12
3607: positiveSegments: 32, negativeSegments: 0
3608: positiveSegments: 4, negativeSegments: 3
3609: positiveSegments: 14, negativeSegments: 0
3611: positiveSegments: 2, negativeSegments: 3
3614: positiveSegments: 8, negativeSegments: 0
3616: positiveSegments: 27, negativeSegments: 0
3618: positiveSegments: 0, negativeSegments: 9
3620: positiveSegments: 6, negativeSegments: 3
3621: positiveSegments: 22, negativeSegments: 6
3623: positiveSegments: 17, negativeSegments: 0
3625: positiveSegments: 58, negativeSegments: 15
3628: positiveSegments: 7, negativeSegments: 3
3629: positiveSegments: 0, negativeSegments: 3
3631: positiveSegments: 25, negativeSegments: 0
3642: positiveSegments: 0, negativeSegments: 12
3648: positiveSegments: 5, negativeSegments: 0
3652: positiveSegments: 4, negativeSegments: 18
3656: positiveSegments: 4, negativeSegments: 21
3658: positiveSegments: 0, negativeSegments: 9
3660: positiveSegments: 13, negativeSegments: 0
3662: positiveSegments: 2, negativeSegments: 9
3663: positiveSegments: 2, negativeSegments: 9
3668: positiveSegments: 0, negativeSegments: 15
3669: positiveSegments: 14, negativeSegments: 0
3672: positiveSegments: 7, negativeSegments: 0
3674: positiveSegments: 0, negativeSegments: 12
3677: positiveSegments: 4, negativeSegments: 6
3678: positiveSegments: 4, negativeSegments: 3
3680: exit early, no segments to save
3680: positiveSegments: 0, negativeSegments: 0
3682: positiveSegments: 13, negativeSegments: 0
3686: positiveSegments: 21, negativeSegments: 6
3687: positiveSegments: 7, negativeSegments: 6
3688: positiveSegments: 2, negativeSegments: 0
3689: positiveSegments: 16, negativeSegments: 3
3690: positiveSegments: 4, negativeSegments: 3
3691: exit early, no segments to save
3691: positiveSegments: 0, negativeSegments: 0
count processed: 1800, current case index: 3694
3694: positiveSegments: 16, negativeSegments: 9
3697: positiveSegments: 0, negativeSegments: 9
3699: positiveSegments: 11, negativeSegments: 6
3700: positiveSegments: 16, negativeSegments: 3
3702: positiveSegments: 8, negativeSegments: 6
3703: positiveSegments: 4, negativeSegments: 6
3704: positiveSegments: 4, negativeSegments: 6
3706: positiveSegments: 3, negativeSegments: 18
3709: positiveSegments: 2, negativeSegments: 0
3710: positiveSegments: 4, negativeSegments: 6
3711: positiveSegments: 19, negativeSegments: 6
3712: positiveSegments: 10, negativeSegments: 6
3713: positiveSegments: 31, negativeSegments: 0
3719: positiveSegments: 18, negativeSegments: 3
3722: positiveSegments: 12, negativeSegments: 6
3723: positiveSegments: 1, negativeSegments: 3
3724: positiveSegments: 5, negativeSegments: 0
3725: positiveSegments: 14, negativeSegments: 3
3727: positiveSegments: 2, negativeSegments: 3
3728: positiveSegments: 11, negativeSegments: 3
3729: positiveSegments: 0, negativeSegments: 9
3730: positiveSegments: 8, negativeSegments: 3
3732: positiveSegments: 4, negativeSegments: 3
3736: positiveSegments: 10, negativeSegments: 0
3737: positiveSegments: 16, negativeSegments: 6
3739: positiveSegments: 15, negativeSegments: 0
3740: positiveSegments: 4, negativeSegments: 3
3743: positiveSegments: 0, negativeSegments: 9
3744: positiveSegments: 28, negativeSegments: 0
3748: positiveSegments: 8, negativeSegments: 3
3749: positiveSegments: 17, negativeSegments: 12
3750: positiveSegments: 0, negativeSegments: 12
3752: positiveSegments: 4, negativeSegments: 18
3753: positiveSegments: 0, negativeSegments: 27
3757: positiveSegments: 0, negativeSegments: 3
3758: positiveSegments: 6, negativeSegments: 3
3761: positiveSegments: 12, negativeSegments: 3
3763: positiveSegments: 2, negativeSegments: 6
3764: positiveSegments: 0, negativeSegments: 9
3768: positiveSegments: 7, negativeSegments: 9
3774: positiveSegments: 10, negativeSegments: 3
3775: positiveSegments: 19, negativeSegments: 6
3776: positiveSegments: 1, negativeSegments: 3
3777: positiveSegments: 8, negativeSegments: 3
3782: positiveSegments: 7, negativeSegments: 3
3783: positiveSegments: 10, negativeSegments: 0
3784: positiveSegments: 12, negativeSegments: 0
3785: positiveSegments: 31, negativeSegments: 9
3789: positiveSegments: 0, negativeSegments: 3
3791: positiveSegments: 7, negativeSegments: 9
3793: positiveSegments: 0, negativeSegments: 3
3798: positiveSegments: 6, negativeSegments: 15
3799: positiveSegments: 4, negativeSegments: 0
3800: positiveSegments: 36, negativeSegments: 3
3802: positiveSegments: 8, negativeSegments: 0
3803: positiveSegments: 12, negativeSegments: 0
3805: positiveSegments: 17, negativeSegments: 6
3810: positiveSegments: 4, negativeSegments: 6
3812: positiveSegments: 14, negativeSegments: 0
3813: positiveSegments: 8, negativeSegments: 6
3814: positiveSegments: 0, negativeSegments: 6
3816: positiveSegments: 15, negativeSegments: 3
3817: positiveSegments: 10, negativeSegments: 6
3818: positiveSegments: 0, negativeSegments: 15
3819: positiveSegments: 13, negativeSegments: 3
3821: exit early, no segments to save
3821: positiveSegments: 0, negativeSegments: 0
3822: positiveSegments: 17, negativeSegments: 0
3823: positiveSegments: 4, negativeSegments: 9
3824: positiveSegments: 37, negativeSegments: 6
3825: positiveSegments: 10, negativeSegments: 0
3828: positiveSegments: 8, negativeSegments: 3
3831: positiveSegments: 7, negativeSegments: 9
3832: positiveSegments: 3, negativeSegments: 18
3835: positiveSegments: 3, negativeSegments: 12
3836: positiveSegments: 17, negativeSegments: 6
3837: positiveSegments: 4, negativeSegments: 9
3839: positiveSegments: 6, negativeSegments: 0
3840: positiveSegments: 8, negativeSegments: 3
3842: positiveSegments: 16, negativeSegments: 18
3843: positiveSegments: 26, negativeSegments: 6
3844: positiveSegments: 10, negativeSegments: 18
3845: positiveSegments: 6, negativeSegments: 6
3846: positiveSegments: 12, negativeSegments: 9
3848: positiveSegments: 6, negativeSegments: 0
3849: positiveSegments: 8, negativeSegments: 3
3850: positiveSegments: 4, negativeSegments: 12
3854: positiveSegments: 51, negativeSegments: 6
3855: positiveSegments: 3, negativeSegments: 6
3857: positiveSegments: 14, negativeSegments: 3
3858: exit early, no segments to save
3858: positiveSegments: 0, negativeSegments: 0
3859: positiveSegments: 2, negativeSegments: 0
3863: positiveSegments: 13, negativeSegments: 15
3864: positiveSegments: 9, negativeSegments: 3
3868: positiveSegments: 5, negativeSegments: 3
3870: positiveSegments: 28, negativeSegments: 12
3877: positiveSegments: 11, negativeSegments: 9
3878: positiveSegments: 22, negativeSegments: 6
3879: positiveSegments: 25, negativeSegments: 0
3881: positiveSegments: 4, negativeSegments: 0
3886: positiveSegments: 12, negativeSegments: 3
count processed: 1900, current case index: 3887
3887: positiveSegments: 5, negativeSegments: 6
3888: positiveSegments: 8, negativeSegments: 6
3889: positiveSegments: 0, negativeSegments: 12
3890: positiveSegments: 10, negativeSegments: 0
3891: positiveSegments: 0, negativeSegments: 6
3893: positiveSegments: 23, negativeSegments: 3
3894: positiveSegments: 2, negativeSegments: 3
3895: positiveSegments: 14, negativeSegments: 9
3897: positiveSegments: 0, negativeSegments: 6
3898: positiveSegments: 8, negativeSegments: 0
3902: positiveSegments: 8, negativeSegments: 6
3904: positiveSegments: 19, negativeSegments: 3
3906: positiveSegments: 8, negativeSegments: 3
3910: positiveSegments: 0, negativeSegments: 3
3912: positiveSegments: 0, negativeSegments: 3
3913: positiveSegments: 21, negativeSegments: 6
3919: positiveSegments: 13, negativeSegments: 3
3922: positiveSegments: 4, negativeSegments: 0
3925: positiveSegments: 0, negativeSegments: 9
3928: positiveSegments: 6, negativeSegments: 0
3929: positiveSegments: 11, negativeSegments: 3
3930: positiveSegments: 25, negativeSegments: 3
3931: positiveSegments: 12, negativeSegments: 3
3934: positiveSegments: 3, negativeSegments: 3
3935: positiveSegments: 9, negativeSegments: 0
3936: positiveSegments: 13, negativeSegments: 0
3937: positiveSegments: 8, negativeSegments: 0
3938: positiveSegments: 9, negativeSegments: 3
3944: positiveSegments: 10, negativeSegments: 15
3949: positiveSegments: 8, negativeSegments: 6
3950: positiveSegments: 11, negativeSegments: 15
3955: positiveSegments: 30, negativeSegments: 0
3958: positiveSegments: 4, negativeSegments: 6
3962: positiveSegments: 15, negativeSegments: 0
3963: positiveSegments: 5, negativeSegments: 9
3967: positiveSegments: 4, negativeSegments: 6
3968: positiveSegments: 10, negativeSegments: 3
3971: nothing saved, all segments filtered
3971: positiveSegments: 1, negativeSegments: 0
3972: positiveSegments: 11, negativeSegments: 0
3973: positiveSegments: 4, negativeSegments: 12
3974: positiveSegments: 0, negativeSegments: 15
3975: positiveSegments: 4, negativeSegments: 9
3976: positiveSegments: 8, negativeSegments: 12
3978: positiveSegments: 4, negativeSegments: 0
3980: exit early, no segments to save
3980: positiveSegments: 0, negativeSegments: 0
3981: positiveSegments: 6, negativeSegments: 0
3986: positiveSegments: 4, negativeSegments: 3
3987: positiveSegments: 3, negativeSegments: 6
3988: positiveSegments: 14, negativeSegments: 3
3990: positiveSegments: 2, negativeSegments: 0
3991: positiveSegments: 0, negativeSegments: 12
3992: exit early, no segments to save
3992: positiveSegments: 0, negativeSegments: 0
3993: positiveSegments: 14, negativeSegments: 9
3994: positiveSegments: 0, negativeSegments: 24
3998: positiveSegments: 0, negativeSegments: 21
3999: positiveSegments: 0, negativeSegments: 3
4000: positiveSegments: 6, negativeSegments: 0
4004: positiveSegments: 0, negativeSegments: 6
4005: positiveSegments: 15, negativeSegments: 9
4007: positiveSegments: 8, negativeSegments: 3
4009: positiveSegments: 1, negativeSegments: 0
4010: positiveSegments: 7, negativeSegments: 3
4011: positiveSegments: 0, negativeSegments: 3
4012: positiveSegments: 4, negativeSegments: 3
4013: positiveSegments: 18, negativeSegments: 9
4016: positiveSegments: 4, negativeSegments: 0
4017: positiveSegments: 5, negativeSegments: 3
4020: positiveSegments: 0, negativeSegments: 6
4022: positiveSegments: 5, negativeSegments: 0
4024: positiveSegments: 6, negativeSegments: 0
4026: positiveSegments: 4, negativeSegments: 21
4027: positiveSegments: 0, negativeSegments: 12
4028: positiveSegments: 0, negativeSegments: 6
4030: positiveSegments: 0, negativeSegments: 9
4032: positiveSegments: 8, negativeSegments: 9
4033: positiveSegments: 16, negativeSegments: 0
4034: positiveSegments: 9, negativeSegments: 0
4035: positiveSegments: 7, negativeSegments: 3
4036: positiveSegments: 9, negativeSegments: 9
4037: exit early, no segments to save
4037: positiveSegments: 0, negativeSegments: 0
4040: positiveSegments: 14, negativeSegments: 3
4042: positiveSegments: 0, negativeSegments: 9
4043: positiveSegments: 27, negativeSegments: 9
4045: positiveSegments: 13, negativeSegments: 0
4046: positiveSegments: 5, negativeSegments: 3
4047: exit early, no segments to save
4047: positiveSegments: 0, negativeSegments: 0
4048: positiveSegments: 7, negativeSegments: 0
4050: positiveSegments: 2, negativeSegments: 12
4054: positiveSegments: 26, negativeSegments: 6
4060: positiveSegments: 8, negativeSegments: 15
4062: positiveSegments: 4, negativeSegments: 0
4066: positiveSegments: 22, negativeSegments: 3
4067: positiveSegments: 3, negativeSegments: 3
4069: exit early, no segments to save
4069: positiveSegments: 0, negativeSegments: 0
4070: positiveSegments: 20, negativeSegments: 0
4072: positiveSegments: 21, negativeSegments: 0
4073: positiveSegments: 0, negativeSegments: 6
4074: positiveSegments: 0, negativeSegments: 15
4077: positiveSegments: 4, negativeSegments: 3
4083: positiveSegments: 0, negativeSegments: 6
count processed: 2000, current case index: 4091
4091: positiveSegments: 9, negativeSegments: 12
4093: positiveSegments: 27, negativeSegments: 6
4098: positiveSegments: 4, negativeSegments: 15
4100: positiveSegments: 1, negativeSegments: 0
4101: positiveSegments: 4, negativeSegments: 3
4106: positiveSegments: 4, negativeSegments: 3
4107: positiveSegments: 4, negativeSegments: 0
4109: positiveSegments: 8, negativeSegments: 0
4111: positiveSegments: 10, negativeSegments: 0
4112: positiveSegments: 8, negativeSegments: 0
4114: positiveSegments: 27, negativeSegments: 6
4115: positiveSegments: 4, negativeSegments: 9
4116: positiveSegments: 8, negativeSegments: 3
4120: positiveSegments: 0, negativeSegments: 3
4122: exit early, no segments to save
4122: positiveSegments: 0, negativeSegments: 0
4127: positiveSegments: 8, negativeSegments: 3
4133: positiveSegments: 0, negativeSegments: 12
4137: positiveSegments: 22, negativeSegments: 3
4140: positiveSegments: 41, negativeSegments: 9
4142: exit early, no segments to save
4142: positiveSegments: 0, negativeSegments: 0
4143: positiveSegments: 29, negativeSegments: 12
4144: positiveSegments: 22, negativeSegments: 3
4146: positiveSegments: 4, negativeSegments: 18
4148: positiveSegments: 8, negativeSegments: 9
4149: positiveSegments: 17, negativeSegments: 0
4150: positiveSegments: 17, negativeSegments: 0
4152: positiveSegments: 6, negativeSegments: 0
4155: positiveSegments: 6, negativeSegments: 3
4162: positiveSegments: 11, negativeSegments: 9
4165: exit early, no segments to save
4165: positiveSegments: 0, negativeSegments: 0
4166: positiveSegments: 30, negativeSegments: 21
4167: positiveSegments: 21, negativeSegments: 0
4168: positiveSegments: 7, negativeSegments: 0
4172: positiveSegments: 9, negativeSegments: 6
4173: positiveSegments: 4, negativeSegments: 3
4177: positiveSegments: 6, negativeSegments: 0
4179: positiveSegments: 26, negativeSegments: 0
4181: positiveSegments: 4, negativeSegments: 3
4183: positiveSegments: 4, negativeSegments: 0
4186: positiveSegments: 7, negativeSegments: 3
4187: exit early, no segments to save
4187: positiveSegments: 0, negativeSegments: 0
4189: positiveSegments: 4, negativeSegments: 9
4191: positiveSegments: 12, negativeSegments: 15
4195: positiveSegments: 0, negativeSegments: 3
4198: positiveSegments: 0, negativeSegments: 3
4201: positiveSegments: 8, negativeSegments: 0
4202: positiveSegments: 8, negativeSegments: 9
4203: positiveSegments: 4, negativeSegments: 0
4206: positiveSegments: 16, negativeSegments: 0
4207: positiveSegments: 8, negativeSegments: 18
4208: positiveSegments: 8, negativeSegments: 0
4210: positiveSegments: 13, negativeSegments: 3
4211: positiveSegments: 18, negativeSegments: 3
4212: positiveSegments: 1, negativeSegments: 9
4213: positiveSegments: 0, negativeSegments: 3
4214: positiveSegments: 1, negativeSegments: 15
4216: positiveSegments: 0, negativeSegments: 18
4219: exit early, no segments to save
4219: positiveSegments: 0, negativeSegments: 0
4222: positiveSegments: 4, negativeSegments: 12
4223: positiveSegments: 2, negativeSegments: 3
4225: positiveSegments: 22, negativeSegments: 12
4227: positiveSegments: 4, negativeSegments: 3
4233: positiveSegments: 0, negativeSegments: 9
4236: positiveSegments: 12, negativeSegments: 3
4238: positiveSegments: 10, negativeSegments: 9
4240: positiveSegments: 32, negativeSegments: 0
4241: positiveSegments: 0, negativeSegments: 3
4242: positiveSegments: 8, negativeSegments: 3
4245: positiveSegments: 26, negativeSegments: 0
4247: positiveSegments: 0, negativeSegments: 3
4249: positiveSegments: 18, negativeSegments: 3
4251: positiveSegments: 17, negativeSegments: 0
4252: positiveSegments: 0, negativeSegments: 12
4253: positiveSegments: 4, negativeSegments: 0
4254: positiveSegments: 4, negativeSegments: 3
4255: positiveSegments: 6, negativeSegments: 3
4256: positiveSegments: 46, negativeSegments: 6
4258: positiveSegments: 17, negativeSegments: 3
4259: positiveSegments: 5, negativeSegments: 6
4262: positiveSegments: 7, negativeSegments: 0
4264: positiveSegments: 18, negativeSegments: 6
4265: positiveSegments: 13, negativeSegments: 0
4268: exit early, no segments to save
4268: positiveSegments: 0, negativeSegments: 0
4269: positiveSegments: 17, negativeSegments: 6
4272: positiveSegments: 23, negativeSegments: 3
4277: exit early, no segments to save
4277: positiveSegments: 0, negativeSegments: 0
4278: positiveSegments: 18, negativeSegments: 3
4279: positiveSegments: 5, negativeSegments: 3
4280: positiveSegments: 11, negativeSegments: 3
4281: positiveSegments: 2, negativeSegments: 3
4282: positiveSegments: 14, negativeSegments: 3
4283: positiveSegments: 28, negativeSegments: 0
4284: positiveSegments: 15, negativeSegments: 9
4286: positiveSegments: 4, negativeSegments: 3
4287: positiveSegments: 0, negativeSegments: 9
4289: positiveSegments: 36, negativeSegments: 6
4290: positiveSegments: 4, negativeSegments: 0
4292: positiveSegments: 8, negativeSegments: 15
4293: positiveSegments: 8, negativeSegments: 0
4294: positiveSegments: 0, negativeSegments: 9
count processed: 2100, current case index: 4296
4296: positiveSegments: 4, negativeSegments: 6
4302: positiveSegments: 25, negativeSegments: 6
4304: positiveSegments: 36, negativeSegments: 0
4305: positiveSegments: 13, negativeSegments: 0
4307: positiveSegments: 6, negativeSegments: 3
4308: positiveSegments: 4, negativeSegments: 3
4309: positiveSegments: 11, negativeSegments: 3
4310: positiveSegments: 4, negativeSegments: 12
4314: positiveSegments: 3, negativeSegments: 9
4316: positiveSegments: 20, negativeSegments: 3
4317: positiveSegments: 13, negativeSegments: 0
4320: positiveSegments: 12, negativeSegments: 6
4322: positiveSegments: 4, negativeSegments: 9
4325: positiveSegments: 3, negativeSegments: 3
4326: positiveSegments: 8, negativeSegments: 3
4327: positiveSegments: 13, negativeSegments: 0
4328: exit early, no segments to save
4328: positiveSegments: 0, negativeSegments: 0
4332: positiveSegments: 13, negativeSegments: 0
4333: positiveSegments: 14, negativeSegments: 3
4335: positiveSegments: 3, negativeSegments: 12
4339: positiveSegments: 7, negativeSegments: 3
4341: positiveSegments: 17, negativeSegments: 12
4345: positiveSegments: 10, negativeSegments: 0
4347: positiveSegments: 26, negativeSegments: 3
4350: positiveSegments: 0, negativeSegments: 3
4352: positiveSegments: 2, negativeSegments: 0
4354: positiveSegments: 12, negativeSegments: 3
4356: positiveSegments: 6, negativeSegments: 9
4362: positiveSegments: 4, negativeSegments: 0
4364: positiveSegments: 9, negativeSegments: 3
4367: positiveSegments: 5, negativeSegments: 12
4368: positiveSegments: 16, negativeSegments: 6
4371: exit early, no segments to save
4371: positiveSegments: 0, negativeSegments: 0
4375: positiveSegments: 0, negativeSegments: 9
4377: positiveSegments: 8, negativeSegments: 15
4380: positiveSegments: 4, negativeSegments: 0
4382: positiveSegments: 14, negativeSegments: 3
4383: positiveSegments: 19, negativeSegments: 6
4385: positiveSegments: 7, negativeSegments: 6
4387: positiveSegments: 0, negativeSegments: 3
4388: positiveSegments: 12, negativeSegments: 0
4389: positiveSegments: 0, negativeSegments: 6
4390: positiveSegments: 12, negativeSegments: 6
4392: positiveSegments: 15, negativeSegments: 3
4396: positiveSegments: 0, negativeSegments: 3
4398: positiveSegments: 26, negativeSegments: 0
4400: positiveSegments: 4, negativeSegments: 6
4401: positiveSegments: 10, negativeSegments: 6
4402: positiveSegments: 7, negativeSegments: 9
4405: positiveSegments: 45, negativeSegments: 0
4406: positiveSegments: 11, negativeSegments: 0
4408: positiveSegments: 5, negativeSegments: 0
4409: positiveSegments: 8, negativeSegments: 9
4411: positiveSegments: 14, negativeSegments: 0
4414: exit early, no segments to save
4414: positiveSegments: 0, negativeSegments: 0
4417: positiveSegments: 5, negativeSegments: 3
4424: positiveSegments: 0, negativeSegments: 9
4425: exit early, no segments to save
4425: positiveSegments: 0, negativeSegments: 0
4426: positiveSegments: 0, negativeSegments: 6
4428: positiveSegments: 4, negativeSegments: 0
4429: positiveSegments: 18, negativeSegments: 6
4430: positiveSegments: 2, negativeSegments: 6
4432: positiveSegments: 8, negativeSegments: 6
4433: exit early, no segments to save
4433: positiveSegments: 0, negativeSegments: 0
4435: positiveSegments: 4, negativeSegments: 0
4437: positiveSegments: 7, negativeSegments: 3
4439: positiveSegments: 13, negativeSegments: 0
4443: positiveSegments: 0, negativeSegments: 6
4449: positiveSegments: 0, negativeSegments: 6
4451: positiveSegments: 12, negativeSegments: 6
4453: positiveSegments: 14, negativeSegments: 0
4456: positiveSegments: 7, negativeSegments: 12
4457: positiveSegments: 8, negativeSegments: 9
4458: positiveSegments: 7, negativeSegments: 0
4459: exit early, no segments to save
4459: positiveSegments: 0, negativeSegments: 0
4461: positiveSegments: 9, negativeSegments: 15
4462: positiveSegments: 11, negativeSegments: 0
4463: positiveSegments: 5, negativeSegments: 12
4464: exit early, no segments to save
4464: positiveSegments: 0, negativeSegments: 0
4466: exit early, no segments to save
4466: positiveSegments: 0, negativeSegments: 0
4470: positiveSegments: 4, negativeSegments: 6
4472: positiveSegments: 13, negativeSegments: 0
4474: positiveSegments: 0, negativeSegments: 9
4475: positiveSegments: 8, negativeSegments: 9
4476: positiveSegments: 19, negativeSegments: 3
4477: positiveSegments: 5, negativeSegments: 0
4478: positiveSegments: 4, negativeSegments: 12
4480: positiveSegments: 4, negativeSegments: 3
4481: positiveSegments: 26, negativeSegments: 6
4483: positiveSegments: 0, negativeSegments: 6
4485: positiveSegments: 1, negativeSegments: 0
4489: positiveSegments: 8, negativeSegments: 9
4490: positiveSegments: 9, negativeSegments: 0
4496: positiveSegments: 17, negativeSegments: 9
4497: positiveSegments: 5, negativeSegments: 0
4498: positiveSegments: 2, negativeSegments: 9
4501: exit early, no segments to save
4501: positiveSegments: 0, negativeSegments: 0
4502: positiveSegments: 8, negativeSegments: 3
4503: positiveSegments: 0, negativeSegments: 3
4504: positiveSegments: 54, negativeSegments: 0
count processed: 2200, current case index: 4509
4509: positiveSegments: 4, negativeSegments: 9
4510: positiveSegments: 21, negativeSegments: 0
4515: positiveSegments: 7, negativeSegments: 0
4519: exit early, no segments to save
4519: positiveSegments: 0, negativeSegments: 0
4520: positiveSegments: 4, negativeSegments: 3
4522: positiveSegments: 0, negativeSegments: 3
4525: positiveSegments: 0, negativeSegments: 9
4530: positiveSegments: 8, negativeSegments: 6
4536: positiveSegments: 2, negativeSegments: 3
4538: positiveSegments: 6, negativeSegments: 3
4540: positiveSegments: 0, negativeSegments: 12
4541: positiveSegments: 4, negativeSegments: 0
4546: positiveSegments: 7, negativeSegments: 3
4547: positiveSegments: 0, negativeSegments: 6
4549: positiveSegments: 10, negativeSegments: 3
4550: positiveSegments: 2, negativeSegments: 9
4557: positiveSegments: 4, negativeSegments: 0
4559: positiveSegments: 12, negativeSegments: 0
4561: positiveSegments: 8, negativeSegments: 0
4564: positiveSegments: 4, negativeSegments: 3
4568: positiveSegments: 15, negativeSegments: 6
4569: positiveSegments: 1, negativeSegments: 3
4572: positiveSegments: 3, negativeSegments: 0
4573: positiveSegments: 40, negativeSegments: 3
4574: positiveSegments: 28, negativeSegments: 12
4576: positiveSegments: 4, negativeSegments: 6
4577: positiveSegments: 0, negativeSegments: 3
4578: positiveSegments: 5, negativeSegments: 0
4579: positiveSegments: 5, negativeSegments: 0
4581: positiveSegments: 4, negativeSegments: 3
4584: positiveSegments: 0, negativeSegments: 9
4589: positiveSegments: 8, negativeSegments: 9
4591: positiveSegments: 19, negativeSegments: 0
4596: positiveSegments: 0, negativeSegments: 6
4597: positiveSegments: 16, negativeSegments: 0
4599: positiveSegments: 10, negativeSegments: 0
4600: positiveSegments: 11, negativeSegments: 6
4602: positiveSegments: 15, negativeSegments: 0
4603: positiveSegments: 4, negativeSegments: 9
4604: positiveSegments: 18, negativeSegments: 12
4605: exit early, no segments to save
4605: positiveSegments: 0, negativeSegments: 0
4607: positiveSegments: 16, negativeSegments: 0
4609: positiveSegments: 0, negativeSegments: 24
4612: positiveSegments: 11, negativeSegments: 6
4613: exit early, no segments to save
4613: positiveSegments: 0, negativeSegments: 0
4616: positiveSegments: 4, negativeSegments: 0
4617: positiveSegments: 9, negativeSegments: 9
4618: positiveSegments: 10, negativeSegments: 6
4619: positiveSegments: 8, negativeSegments: 6
4620: positiveSegments: 6, negativeSegments: 0
4621: positiveSegments: 14, negativeSegments: 6
4622: positiveSegments: 4, negativeSegments: 0
4626: positiveSegments: 8, negativeSegments: 3
4627: positiveSegments: 13, negativeSegments: 0
4631: positiveSegments: 3, negativeSegments: 3
4632: positiveSegments: 18, negativeSegments: 6
4635: positiveSegments: 8, negativeSegments: 0
4639: positiveSegments: 6, negativeSegments: 6
4640: positiveSegments: 4, negativeSegments: 0
4644: positiveSegments: 18, negativeSegments: 15
4646: positiveSegments: 9, negativeSegments: 3
4648: exit early, no segments to save
4648: positiveSegments: 0, negativeSegments: 0
4650: positiveSegments: 3, negativeSegments: 0
4652: positiveSegments: 31, negativeSegments: 6
4653: positiveSegments: 11, negativeSegments: 3
4654: positiveSegments: 9, negativeSegments: 9
4655: positiveSegments: 4, negativeSegments: 0
4656: positiveSegments: 9, negativeSegments: 0
4657: positiveSegments: 26, negativeSegments: 0
4658: positiveSegments: 8, negativeSegments: 6
4660: positiveSegments: 4, negativeSegments: 6
4662: positiveSegments: 16, negativeSegments: 0
4665: positiveSegments: 19, negativeSegments: 3
4666: positiveSegments: 5, negativeSegments: 15
4670: positiveSegments: 48, negativeSegments: 0
4673: exit early, no segments to save
4673: positiveSegments: 0, negativeSegments: 0
4678: positiveSegments: 0, negativeSegments: 6
4683: positiveSegments: 9, negativeSegments: 0
4684: positiveSegments: 25, negativeSegments: 9
4686: positiveSegments: 11, negativeSegments: 3
4690: exit early, no segments to save
4690: positiveSegments: 0, negativeSegments: 0
4695: positiveSegments: 6, negativeSegments: 0
4700: positiveSegments: 10, negativeSegments: 3
4702: exit early, no segments to save
4702: positiveSegments: 0, negativeSegments: 0
4703: exit early, no segments to save
4703: positiveSegments: 0, negativeSegments: 0
4705: positiveSegments: 4, negativeSegments: 0
4706: positiveSegments: 0, negativeSegments: 9
4711: positiveSegments: 8, negativeSegments: 3
4713: positiveSegments: 0, negativeSegments: 12
4714: positiveSegments: 6, negativeSegments: 3
4715: positiveSegments: 20, negativeSegments: 9
4716: positiveSegments: 4, negativeSegments: 3
4717: positiveSegments: 0, negativeSegments: 6
4718: positiveSegments: 15, negativeSegments: 3
4721: positiveSegments: 28, negativeSegments: 0
4724: positiveSegments: 0, negativeSegments: 3
4726: positiveSegments: 5, negativeSegments: 3
4727: nothing saved, all segments filtered
4727: positiveSegments: 1, negativeSegments: 0
4729: positiveSegments: 4, negativeSegments: 6
4731: positiveSegments: 8, negativeSegments: 12
count processed: 2300, current case index: 4732
4732: positiveSegments: 1, negativeSegments: 15
4733: exit early, no segments to save
4733: positiveSegments: 0, negativeSegments: 0
4739: positiveSegments: 13, negativeSegments: 0
4741: exit early, no segments to save
4741: positiveSegments: 0, negativeSegments: 0
4743: positiveSegments: 31, negativeSegments: 3
4744: positiveSegments: 6, negativeSegments: 0
4745: positiveSegments: 4, negativeSegments: 9
4746: positiveSegments: 12, negativeSegments: 6
4749: positiveSegments: 22, negativeSegments: 3
4750: positiveSegments: 0, negativeSegments: 12
4752: positiveSegments: 7, negativeSegments: 9
4755: positiveSegments: 4, negativeSegments: 0
4757: positiveSegments: 18, negativeSegments: 15
4759: positiveSegments: 28, negativeSegments: 9
4761: positiveSegments: 6, negativeSegments: 0
4763: positiveSegments: 6, negativeSegments: 0
4764: positiveSegments: 12, negativeSegments: 12
4767: positiveSegments: 4, negativeSegments: 3
4768: positiveSegments: 7, negativeSegments: 3
4769: exit early, no segments to save
4769: positiveSegments: 0, negativeSegments: 0
4771: positiveSegments: 5, negativeSegments: 0
4773: positiveSegments: 13, negativeSegments: 6
4775: positiveSegments: 19, negativeSegments: 3
4777: positiveSegments: 10, negativeSegments: 0
4778: positiveSegments: 17, negativeSegments: 0
4779: positiveSegments: 27, negativeSegments: 0
4780: positiveSegments: 8, negativeSegments: 3
4781: positiveSegments: 4, negativeSegments: 21
4783: positiveSegments: 4, negativeSegments: 0
4784: exit early, no segments to save
4784: positiveSegments: 0, negativeSegments: 0
4785: positiveSegments: 0, negativeSegments: 6
4786: positiveSegments: 7, negativeSegments: 0
4788: positiveSegments: 13, negativeSegments: 3
4789: positiveSegments: 8, negativeSegments: 6
4790: positiveSegments: 2, negativeSegments: 0
4792: positiveSegments: 4, negativeSegments: 9
4794: positiveSegments: 0, negativeSegments: 3
4798: positiveSegments: 10, negativeSegments: 9
4800: positiveSegments: 26, negativeSegments: 3
4801: positiveSegments: 29, negativeSegments: 9
4802: positiveSegments: 9, negativeSegments: 0
4803: positiveSegments: 4, negativeSegments: 3
4805: positiveSegments: 4, negativeSegments: 0
4806: positiveSegments: 0, negativeSegments: 9
4808: positiveSegments: 18, negativeSegments: 3
4809: positiveSegments: 16, negativeSegments: 6
4813: positiveSegments: 4, negativeSegments: 3
4816: positiveSegments: 17, negativeSegments: 0
4817: positiveSegments: 45, negativeSegments: 3
4818: positiveSegments: 0, negativeSegments: 3
4820: positiveSegments: 16, negativeSegments: 0
4823: positiveSegments: 5, negativeSegments: 6
4825: positiveSegments: 10, negativeSegments: 0
4826: positiveSegments: 8, negativeSegments: 3
4828: positiveSegments: 22, negativeSegments: 0
4830: positiveSegments: 4, negativeSegments: 0
4831: positiveSegments: 12, negativeSegments: 3
4834: nothing saved, all segments filtered
4834: positiveSegments: 3, negativeSegments: 0
4835: positiveSegments: 11, negativeSegments: 6
4836: nothing saved, all segments filtered
4836: positiveSegments: 11, negativeSegments: 0
4837: positiveSegments: 3, negativeSegments: 3
4839: positiveSegments: 16, negativeSegments: 6
4841: positiveSegments: 7, negativeSegments: 0
4843: positiveSegments: 3, negativeSegments: 6
4844: positiveSegments: 0, negativeSegments: 3
4847: positiveSegments: 0, negativeSegments: 3
4851: positiveSegments: 23, negativeSegments: 0
4853: positiveSegments: 11, negativeSegments: 9
4857: positiveSegments: 0, negativeSegments: 3
4858: positiveSegments: 22, negativeSegments: 3
4859: positiveSegments: 0, negativeSegments: 6
4860: positiveSegments: 3, negativeSegments: 0
4861: positiveSegments: 20, negativeSegments: 0
4865: positiveSegments: 3, negativeSegments: 9
4869: positiveSegments: 16, negativeSegments: 3
4871: positiveSegments: 34, negativeSegments: 3
4872: positiveSegments: 4, negativeSegments: 6
4874: positiveSegments: 0, negativeSegments: 6
4875: positiveSegments: 5, negativeSegments: 3
4879: positiveSegments: 8, negativeSegments: 3
4880: positiveSegments: 8, negativeSegments: 3
4882: positiveSegments: 4, negativeSegments: 3
4883: positiveSegments: 4, negativeSegments: 0
4886: positiveSegments: 4, negativeSegments: 0
4887: positiveSegments: 21, negativeSegments: 15
4893: positiveSegments: 27, negativeSegments: 0
4894: positiveSegments: 13, negativeSegments: 6
4897: positiveSegments: 16, negativeSegments: 3
4899: positiveSegments: 0, negativeSegments: 12
4901: positiveSegments: 3, negativeSegments: 0
4902: positiveSegments: 13, negativeSegments: 3
4903: positiveSegments: 0, negativeSegments: 3
4904: positiveSegments: 0, negativeSegments: 3
4907: positiveSegments: 6, negativeSegments: 6
4911: positiveSegments: 14, negativeSegments: 6
4912: positiveSegments: 12, negativeSegments: 15
4913: positiveSegments: 0, negativeSegments: 3
4914: positiveSegments: 7, negativeSegments: 0
4916: positiveSegments: 18, negativeSegments: 0
4925: positiveSegments: 0, negativeSegments: 6
count processed: 2400, current case index: 4929
4929: positiveSegments: 10, negativeSegments: 3
4932: positiveSegments: 0, negativeSegments: 6
4933: positiveSegments: 21, negativeSegments: 0
4934: positiveSegments: 28, negativeSegments: 12
4935: positiveSegments: 2, negativeSegments: 6
4936: exit early, no segments to save
4936: positiveSegments: 0, negativeSegments: 0
4938: exit early, no segments to save
4938: positiveSegments: 0, negativeSegments: 0
4939: positiveSegments: 4, negativeSegments: 9
4941: nothing saved, all segments filtered
4941: positiveSegments: 1, negativeSegments: 0
4942: positiveSegments: 18, negativeSegments: 3
4943: positiveSegments: 0, negativeSegments: 9
4946: positiveSegments: 4, negativeSegments: 0
4947: positiveSegments: 4, negativeSegments: 18
4949: positiveSegments: 0, negativeSegments: 6
4951: positiveSegments: 8, negativeSegments: 6
4953: positiveSegments: 21, negativeSegments: 9
4954: positiveSegments: 11, negativeSegments: 6
4957: positiveSegments: 15, negativeSegments: 9
4958: positiveSegments: 18, negativeSegments: 6
4959: positiveSegments: 35, negativeSegments: 3
4964: positiveSegments: 4, negativeSegments: 0
4965: positiveSegments: 0, negativeSegments: 3
4966: positiveSegments: 24, negativeSegments: 3
4971: positiveSegments: 13, negativeSegments: 0
4973: positiveSegments: 4, negativeSegments: 6
4974: positiveSegments: 10, negativeSegments: 6
4976: positiveSegments: 9, negativeSegments: 9
4977: positiveSegments: 8, negativeSegments: 0
4982: positiveSegments: 7, negativeSegments: 3
4985: nothing saved, all segments filtered
4985: positiveSegments: 1, negativeSegments: 0
4987: positiveSegments: 9, negativeSegments: 12
4989: positiveSegments: 6, negativeSegments: 3
4991: positiveSegments: 0, negativeSegments: 3
4992: positiveSegments: 12, negativeSegments: 0
4995: positiveSegments: 7, negativeSegments: 15
4997: positiveSegments: 34, negativeSegments: 3
4998: positiveSegments: 7, negativeSegments: 0
4999: positiveSegments: 11, negativeSegments: 12
5001: positiveSegments: 3, negativeSegments: 0
5004: positiveSegments: 26, negativeSegments: 0
5005: positiveSegments: 16, negativeSegments: 0
5006: positiveSegments: 24, negativeSegments: 3
5008: positiveSegments: 0, negativeSegments: 6
5010: positiveSegments: 12, negativeSegments: 15
5014: positiveSegments: 0, negativeSegments: 15
5015: positiveSegments: 2, negativeSegments: 9
5018: positiveSegments: 13, negativeSegments: 0
5019: positiveSegments: 19, negativeSegments: 15
5021: positiveSegments: 6, negativeSegments: 6
5024: positiveSegments: 7, negativeSegments: 3
5027: positiveSegments: 8, negativeSegments: 0
5029: positiveSegments: 0, negativeSegments: 3
5031: positiveSegments: 6, negativeSegments: 0
5033: positiveSegments: 4, negativeSegments: 3
5036: positiveSegments: 25, negativeSegments: 0
5040: positiveSegments: 7, negativeSegments: 3
5043: positiveSegments: 14, negativeSegments: 6
5044: positiveSegments: 20, negativeSegments: 6
5045: positiveSegments: 12, negativeSegments: 3
5046: positiveSegments: 8, negativeSegments: 0
5052: positiveSegments: 15, negativeSegments: 0
5059: positiveSegments: 24, negativeSegments: 0
5060: positiveSegments: 0, negativeSegments: 18
5061: positiveSegments: 3, negativeSegments: 0
5068: positiveSegments: 8, negativeSegments: 3
5070: positiveSegments: 10, negativeSegments: 0
5072: positiveSegments: 0, negativeSegments: 12
5077: positiveSegments: 4, negativeSegments: 6
5079: positiveSegments: 0, negativeSegments: 6
5080: positiveSegments: 4, negativeSegments: 0
5084: positiveSegments: 13, negativeSegments: 0
5089: positiveSegments: 4, negativeSegments: 9
5090: positiveSegments: 11, negativeSegments: 3
5091: positiveSegments: 0, negativeSegments: 9
5092: positiveSegments: 4, negativeSegments: 3
5093: positiveSegments: 6, negativeSegments: 6
5099: positiveSegments: 28, negativeSegments: 12
5100: positiveSegments: 29, negativeSegments: 0
5104: positiveSegments: 3, negativeSegments: 6
5106: positiveSegments: 4, negativeSegments: 15
5108: positiveSegments: 15, negativeSegments: 3
5109: positiveSegments: 14, negativeSegments: 3
5110: positiveSegments: 13, negativeSegments: 0
5111: positiveSegments: 4, negativeSegments: 9
5112: positiveSegments: 4, negativeSegments: 9
5116: positiveSegments: 4, negativeSegments: 0
5117: positiveSegments: 2, negativeSegments: 0
5118: positiveSegments: 17, negativeSegments: 0
5119: positiveSegments: 50, negativeSegments: 3
5122: positiveSegments: 3, negativeSegments: 6
5124: positiveSegments: 2, negativeSegments: 3
5125: positiveSegments: 4, negativeSegments: 0
5126: positiveSegments: 10, negativeSegments: 3
5130: exit early, no segments to save
5130: positiveSegments: 0, negativeSegments: 0
5131: positiveSegments: 20, negativeSegments: 3
5135: positiveSegments: 13, negativeSegments: 9
5137: positiveSegments: 4, negativeSegments: 6
5139: positiveSegments: 26, negativeSegments: 0
5140: positiveSegments: 0, negativeSegments: 9
5141: positiveSegments: 2, negativeSegments: 0
count processed: 2500, current case index: 5142
5142: positiveSegments: 3, negativeSegments: 0
5143: positiveSegments: 0, negativeSegments: 3
5148: exit early, no segments to save
5148: positiveSegments: 0, negativeSegments: 0
5149: positiveSegments: 9, negativeSegments: 0
5150: positiveSegments: 0, negativeSegments: 6
5151: positiveSegments: 10, negativeSegments: 0
5152: positiveSegments: 27, negativeSegments: 6
5153: positiveSegments: 7, negativeSegments: 6
5155: positiveSegments: 4, negativeSegments: 3
5156: positiveSegments: 5, negativeSegments: 0
5157: positiveSegments: 4, negativeSegments: 0
5162: positiveSegments: 4, negativeSegments: 9
5163: positiveSegments: 23, negativeSegments: 9
5164: exit early, no segments to save
5164: positiveSegments: 0, negativeSegments: 0
5166: positiveSegments: 16, negativeSegments: 9
5173: positiveSegments: 40, negativeSegments: 0
5174: positiveSegments: 22, negativeSegments: 0
5175: nothing saved, all segments filtered
5175: positiveSegments: 2, negativeSegments: 0
5178: positiveSegments: 13, negativeSegments: 6
5179: nothing saved, all segments filtered
5179: positiveSegments: 2, negativeSegments: 0
5180: positiveSegments: 4, negativeSegments: 12
5182: positiveSegments: 0, negativeSegments: 6
5183: positiveSegments: 4, negativeSegments: 0
5184: positiveSegments: 12, negativeSegments: 3
5185: positiveSegments: 11, negativeSegments: 0
5187: positiveSegments: 17, negativeSegments: 0
5192: positiveSegments: 0, negativeSegments: 6
5193: positiveSegments: 0, negativeSegments: 6
5194: positiveSegments: 5, negativeSegments: 0
5195: positiveSegments: 4, negativeSegments: 3
5198: positiveSegments: 12, negativeSegments: 6
5201: positiveSegments: 37, negativeSegments: 0
5202: positiveSegments: 2, negativeSegments: 6
5203: positiveSegments: 16, negativeSegments: 12
5204: positiveSegments: 12, negativeSegments: 0
5209: exit early, no segments to save
5209: positiveSegments: 0, negativeSegments: 0
5213: positiveSegments: 8, negativeSegments: 6
5214: positiveSegments: 0, negativeSegments: 3
5217: positiveSegments: 9, negativeSegments: 3
5218: exit early, no segments to save
5218: positiveSegments: 0, negativeSegments: 0
5221: positiveSegments: 11, negativeSegments: 0
5222: positiveSegments: 12, negativeSegments: 12
5223: exit early, no segments to save
5223: positiveSegments: 0, negativeSegments: 0
5224: positiveSegments: 12, negativeSegments: 15
5225: positiveSegments: 15, negativeSegments: 0
5226: positiveSegments: 7, negativeSegments: 0
5229: positiveSegments: 14, negativeSegments: 6
5230: positiveSegments: 7, negativeSegments: 0
5232: positiveSegments: 1, negativeSegments: 12
5234: positiveSegments: 8, negativeSegments: 3
5235: positiveSegments: 6, negativeSegments: 21
5237: positiveSegments: 8, negativeSegments: 0
5245: positiveSegments: 33, negativeSegments: 6
5246: positiveSegments: 12, negativeSegments: 0
5247: positiveSegments: 14, negativeSegments: 9
5251: positiveSegments: 11, negativeSegments: 0
5253: positiveSegments: 6, negativeSegments: 0
5254: positiveSegments: 4, negativeSegments: 3
5255: positiveSegments: 0, negativeSegments: 9
5259: positiveSegments: 11, negativeSegments: 0
5264: positiveSegments: 0, negativeSegments: 21
5265: exit early, no segments to save
5265: positiveSegments: 0, negativeSegments: 0
5266: positiveSegments: 6, negativeSegments: 0
5267: positiveSegments: 8, negativeSegments: 6
5270: positiveSegments: 8, negativeSegments: 0
5271: positiveSegments: 0, negativeSegments: 15
5274: exit early, no segments to save
5274: positiveSegments: 0, negativeSegments: 0
5277: positiveSegments: 4, negativeSegments: 12
5278: positiveSegments: 6, negativeSegments: 9
5280: positiveSegments: 5, negativeSegments: 0
5283: positiveSegments: 2, negativeSegments: 3
5284: positiveSegments: 24, negativeSegments: 0
5285: positiveSegments: 13, negativeSegments: 0
5288: positiveSegments: 4, negativeSegments: 3
5292: positiveSegments: 7, negativeSegments: 0
5295: positiveSegments: 16, negativeSegments: 9
5296: positiveSegments: 5, negativeSegments: 3
5297: positiveSegments: 6, negativeSegments: 0
5298: positiveSegments: 32, negativeSegments: 6
5299: positiveSegments: 8, negativeSegments: 0
5301: exit early, no segments to save
5301: positiveSegments: 0, negativeSegments: 0
5302: positiveSegments: 12, negativeSegments: 0
5304: positiveSegments: 29, negativeSegments: 0
5305: positiveSegments: 21, negativeSegments: 6
5308: positiveSegments: 0, negativeSegments: 18
5309: positiveSegments: 0, negativeSegments: 6
5310: positiveSegments: 15, negativeSegments: 0
5311: positiveSegments: 31, negativeSegments: 3
5314: positiveSegments: 0, negativeSegments: 18
5317: positiveSegments: 11, negativeSegments: 15
5319: positiveSegments: 24, negativeSegments: 9
5321: positiveSegments: 0, negativeSegments: 12
5322: positiveSegments: 10, negativeSegments: 6
5323: positiveSegments: 16, negativeSegments: 3
5324: positiveSegments: 8, negativeSegments: 0
5327: nothing saved, all segments filtered
5327: positiveSegments: 4, negativeSegments: 0
5332: positiveSegments: 0, negativeSegments: 3
5339: nothing saved, all segments filtered
5339: positiveSegments: 2, negativeSegments: 0
5342: positiveSegments: 0, negativeSegments: 6
5344: positiveSegments: 4, negativeSegments: 9
count processed: 2600, current case index: 5346
5346: positiveSegments: 33, negativeSegments: 6
5347: positiveSegments: 21, negativeSegments: 12
5348: exit early, no segments to save
5348: positiveSegments: 0, negativeSegments: 0
5349: positiveSegments: 15, negativeSegments: 6
5352: positiveSegments: 8, negativeSegments: 9
5353: positiveSegments: 8, negativeSegments: 6
5356: positiveSegments: 3, negativeSegments: 0
5359: positiveSegments: 4, negativeSegments: 3
5360: positiveSegments: 12, negativeSegments: 21
5362: positiveSegments: 24, negativeSegments: 3
5363: positiveSegments: 15, negativeSegments: 12
5364: positiveSegments: 3, negativeSegments: 6
5366: exit early, no segments to save
5366: positiveSegments: 0, negativeSegments: 0
5367: positiveSegments: 12, negativeSegments: 3
5368: positiveSegments: 8, negativeSegments: 0
5371: positiveSegments: 4, negativeSegments: 6
5372: positiveSegments: 14, negativeSegments: 3
5379: positiveSegments: 11, negativeSegments: 3
5383: positiveSegments: 7, negativeSegments: 6
5387: positiveSegments: 12, negativeSegments: 6
5388: positiveSegments: 0, negativeSegments: 6
5390: positiveSegments: 6, negativeSegments: 0
5393: positiveSegments: 0, negativeSegments: 3
5394: positiveSegments: 4, negativeSegments: 0
5395: positiveSegments: 0, negativeSegments: 3
5396: positiveSegments: 30, negativeSegments: 18
5397: positiveSegments: 4, negativeSegments: 0
5399: positiveSegments: 12, negativeSegments: 3
5402: positiveSegments: 4, negativeSegments: 0
5403: positiveSegments: 11, negativeSegments: 9
5406: positiveSegments: 0, negativeSegments: 3
5411: positiveSegments: 7, negativeSegments: 0
5415: positiveSegments: 13, negativeSegments: 0
5416: positiveSegments: 46, negativeSegments: 3
5420: positiveSegments: 12, negativeSegments: 3
5421: positiveSegments: 43, negativeSegments: 6
5422: nothing saved, all segments filtered
5422: positiveSegments: 2, negativeSegments: 0
5423: positiveSegments: 31, negativeSegments: 0
5425: positiveSegments: 6, negativeSegments: 12
5427: positiveSegments: 8, negativeSegments: 0
5431: positiveSegments: 0, negativeSegments: 3
5434: positiveSegments: 4, negativeSegments: 6
5436: positiveSegments: 3, negativeSegments: 0
5442: positiveSegments: 8, negativeSegments: 0
5443: positiveSegments: 13, negativeSegments: 6
5446: positiveSegments: 9, negativeSegments: 0
5449: positiveSegments: 6, negativeSegments: 0
5451: positiveSegments: 20, negativeSegments: 3
5453: positiveSegments: 6, negativeSegments: 0
5454: positiveSegments: 22, negativeSegments: 0
5456: positiveSegments: 0, negativeSegments: 6
5458: positiveSegments: 42, negativeSegments: 9
5460: positiveSegments: 5, negativeSegments: 0
5462: positiveSegments: 0, negativeSegments: 6
5463: positiveSegments: 7, negativeSegments: 6
5467: positiveSegments: 14, negativeSegments: 3
5474: positiveSegments: 4, negativeSegments: 6
5475: positiveSegments: 4, negativeSegments: 12
5476: exit early, no segments to save
5476: positiveSegments: 0, negativeSegments: 0
5478: positiveSegments: 7, negativeSegments: 0
5479: positiveSegments: 3, negativeSegments: 6
5480: positiveSegments: 4, negativeSegments: 9
5484: positiveSegments: 4, negativeSegments: 6
5486: positiveSegments: 15, negativeSegments: 3
5487: positiveSegments: 4, negativeSegments: 6
5490: positiveSegments: 18, negativeSegments: 0
5492: positiveSegments: 7, negativeSegments: 6
5495: positiveSegments: 8, negativeSegments: 9
5497: positiveSegments: 28, negativeSegments: 3
5499: positiveSegments: 9, negativeSegments: 6
5500: positiveSegments: 2, negativeSegments: 0
5501: exit early, no segments to save
5501: positiveSegments: 0, negativeSegments: 0
5502: positiveSegments: 9, negativeSegments: 0
5505: positiveSegments: 16, negativeSegments: 3
5507: positiveSegments: 4, negativeSegments: 12
5508: positiveSegments: 0, negativeSegments: 12
5509: positiveSegments: 0, negativeSegments: 12
5511: positiveSegments: 10, negativeSegments: 3
5513: positiveSegments: 8, negativeSegments: 9
5515: positiveSegments: 4, negativeSegments: 12
5516: positiveSegments: 29, negativeSegments: 6
5517: positiveSegments: 4, negativeSegments: 6
5519: positiveSegments: 7, negativeSegments: 9
5520: exit early, no segments to save
5520: positiveSegments: 0, negativeSegments: 0
5524: positiveSegments: 14, negativeSegments: 3
5526: positiveSegments: 1, negativeSegments: 6
5531: positiveSegments: 8, negativeSegments: 9
5533: positiveSegments: 0, negativeSegments: 3
5534: positiveSegments: 42, negativeSegments: 15
5536: positiveSegments: 7, negativeSegments: 3
5537: positiveSegments: 6, negativeSegments: 6
5538: positiveSegments: 11, negativeSegments: 0
5541: positiveSegments: 0, negativeSegments: 3
5544: positiveSegments: 2, negativeSegments: 0
5546: positiveSegments: 7, negativeSegments: 0
5548: positiveSegments: 5, negativeSegments: 3
5552: positiveSegments: 27, negativeSegments: 0
5556: positiveSegments: 4, negativeSegments: 0
5561: positiveSegments: 0, negativeSegments: 3
5562: exit early, no segments to save
5562: positiveSegments: 0, negativeSegments: 0
count processed: 2700, current case index: 5564
5564: positiveSegments: 23, negativeSegments: 0
5566: positiveSegments: 6, negativeSegments: 6
5568: positiveSegments: 18, negativeSegments: 3
5571: exit early, no segments to save
5571: positiveSegments: 0, negativeSegments: 0
5572: positiveSegments: 6, negativeSegments: 9
5573: positiveSegments: 7, negativeSegments: 0
5574: positiveSegments: 14, negativeSegments: 9
5578: positiveSegments: 2, negativeSegments: 3
5582: positiveSegments: 12, negativeSegments: 15
5583: positiveSegments: 15, negativeSegments: 3
5585: positiveSegments: 17, negativeSegments: 3
5587: nothing saved, all segments filtered
5587: positiveSegments: 2, negativeSegments: 0
5589: positiveSegments: 4, negativeSegments: 6
5593: positiveSegments: 16, negativeSegments: 3
5594: positiveSegments: 4, negativeSegments: 3
5595: positiveSegments: 31, negativeSegments: 3
5597: positiveSegments: 7, negativeSegments: 0
5598: positiveSegments: 6, negativeSegments: 0
5600: positiveSegments: 13, negativeSegments: 3
5601: positiveSegments: 11, negativeSegments: 6
5602: positiveSegments: 4, negativeSegments: 12
5603: positiveSegments: 8, negativeSegments: 3
5607: positiveSegments: 36, negativeSegments: 9
5608: positiveSegments: 8, negativeSegments: 0
5610: positiveSegments: 7, negativeSegments: 0
5612: positiveSegments: 18, negativeSegments: 3
5613: positiveSegments: 8, negativeSegments: 9
5614: positiveSegments: 16, negativeSegments: 0
5616: positiveSegments: 0, negativeSegments: 9
5617: nothing saved, all segments filtered
5617: positiveSegments: 4, negativeSegments: 0
5618: positiveSegments: 0, negativeSegments: 15
5620: positiveSegments: 18, negativeSegments: 3
5621: positiveSegments: 10, negativeSegments: 12
5624: positiveSegments: 23, negativeSegments: 3
5626: positiveSegments: 19, negativeSegments: 3
5627: positiveSegments: 5, negativeSegments: 15
5629: positiveSegments: 5, negativeSegments: 0
5630: positiveSegments: 14, negativeSegments: 12
5633: positiveSegments: 16, negativeSegments: 0
5635: positiveSegments: 7, negativeSegments: 0
5637: positiveSegments: 11, negativeSegments: 15
5638: positiveSegments: 18, negativeSegments: 3
5641: positiveSegments: 23, negativeSegments: 6
5642: positiveSegments: 11, negativeSegments: 0
5646: positiveSegments: 0, negativeSegments: 3
5647: positiveSegments: 4, negativeSegments: 3
5648: positiveSegments: 13, negativeSegments: 6
5650: positiveSegments: 10, negativeSegments: 3
5654: positiveSegments: 0, negativeSegments: 6
5655: positiveSegments: 14, negativeSegments: 3
5657: positiveSegments: 9, negativeSegments: 3
5658: positiveSegments: 15, negativeSegments: 3
5659: positiveSegments: 0, negativeSegments: 12
5662: positiveSegments: 2, negativeSegments: 0
5664: exit early, no segments to save
5664: positiveSegments: 0, negativeSegments: 0
5665: positiveSegments: 8, negativeSegments: 12
5669: positiveSegments: 8, negativeSegments: 9
5670: positiveSegments: 13, negativeSegments: 9
5671: positiveSegments: 0, negativeSegments: 6
5673: positiveSegments: 19, negativeSegments: 3
5675: positiveSegments: 6, negativeSegments: 9
5677: positiveSegments: 4, negativeSegments: 6
5678: positiveSegments: 19, negativeSegments: 30
5680: positiveSegments: 4, negativeSegments: 6
5682: positiveSegments: 24, negativeSegments: 0
5684: positiveSegments: 9, negativeSegments: 6
5685: positiveSegments: 9, negativeSegments: 6
5687: positiveSegments: 33, negativeSegments: 6
5690: positiveSegments: 7, negativeSegments: 6
5691: positiveSegments: 5, negativeSegments: 0
5692: positiveSegments: 0, negativeSegments: 6
5693: exit early, no segments to save
5693: positiveSegments: 0, negativeSegments: 0
5694: positiveSegments: 6, negativeSegments: 6
5696: positiveSegments: 10, negativeSegments: 0
5698: positiveSegments: 41, negativeSegments: 0
5703: positiveSegments: 8, negativeSegments: 6
5711: exit early, no segments to save
5711: positiveSegments: 0, negativeSegments: 0
5715: positiveSegments: 16, negativeSegments: 18
5717: positiveSegments: 4, negativeSegments: 0
5718: positiveSegments: 4, negativeSegments: 3
5719: positiveSegments: 5, negativeSegments: 3
5721: positiveSegments: 1, negativeSegments: 6
5724: positiveSegments: 0, negativeSegments: 21
5725: positiveSegments: 10, negativeSegments: 9
5727: positiveSegments: 0, negativeSegments: 6
5729: positiveSegments: 2, negativeSegments: 15
5733: positiveSegments: 4, negativeSegments: 15
5734: positiveSegments: 3, negativeSegments: 0
5743: positiveSegments: 8, negativeSegments: 3
5745: positiveSegments: 7, negativeSegments: 0
5746: positiveSegments: 32, negativeSegments: 3
5749: positiveSegments: 10, negativeSegments: 6
5750: positiveSegments: 14, negativeSegments: 0
5751: positiveSegments: 8, negativeSegments: 12
5753: positiveSegments: 6, negativeSegments: 0
5755: positiveSegments: 12, negativeSegments: 0
5759: positiveSegments: 5, negativeSegments: 6
5760: positiveSegments: 27, negativeSegments: 18
5765: positiveSegments: 7, negativeSegments: 0
5769: positiveSegments: 2, negativeSegments: 12
count processed: 2800, current case index: 5771
5771: positiveSegments: 24, negativeSegments: 3
5772: positiveSegments: 9, negativeSegments: 12
5777: positiveSegments: 0, negativeSegments: 9
5780: positiveSegments: 15, negativeSegments: 6
5781: positiveSegments: 12, negativeSegments: 3
5782: positiveSegments: 2, negativeSegments: 3
5783: positiveSegments: 11, negativeSegments: 15
5784: positiveSegments: 11, negativeSegments: 0
5787: positiveSegments: 58, negativeSegments: 0
5788: positiveSegments: 8, negativeSegments: 12
5793: positiveSegments: 0, negativeSegments: 9
5795: positiveSegments: 19, negativeSegments: 6
5799: positiveSegments: 8, negativeSegments: 0
5800: exit early, no segments to save
5800: positiveSegments: 0, negativeSegments: 0
5801: positiveSegments: 0, negativeSegments: 3
5805: positiveSegments: 6, negativeSegments: 0
5806: positiveSegments: 32, negativeSegments: 3
5808: positiveSegments: 7, negativeSegments: 3
5809: positiveSegments: 5, negativeSegments: 21
5810: exit early, no segments to save
5810: positiveSegments: 0, negativeSegments: 0
5811: positiveSegments: 4, negativeSegments: 0
5814: positiveSegments: 8, negativeSegments: 0
5816: positiveSegments: 3, negativeSegments: 9
5817: exit early, no segments to save
5817: positiveSegments: 0, negativeSegments: 0
5819: positiveSegments: 12, negativeSegments: 15
5823: exit early, no segments to save
5823: positiveSegments: 0, negativeSegments: 0
5825: positiveSegments: 11, negativeSegments: 12
5826: positiveSegments: 17, negativeSegments: 15
5827: positiveSegments: 16, negativeSegments: 9
5829: positiveSegments: 8, negativeSegments: 6
5831: positiveSegments: 29, negativeSegments: 3
5832: positiveSegments: 8, negativeSegments: 3
5834: positiveSegments: 0, negativeSegments: 3
5837: positiveSegments: 20, negativeSegments: 6
5839: positiveSegments: 6, negativeSegments: 9
5840: positiveSegments: 4, negativeSegments: 21
5842: positiveSegments: 0, negativeSegments: 9
5843: positiveSegments: 8, negativeSegments: 12
5844: positiveSegments: 14, negativeSegments: 6
5848: positiveSegments: 7, negativeSegments: 3
5849: positiveSegments: 19, negativeSegments: 0
5851: positiveSegments: 0, negativeSegments: 9
5859: positiveSegments: 21, negativeSegments: 9
5860: positiveSegments: 12, negativeSegments: 9
5861: positiveSegments: 7, negativeSegments: 0
5862: positiveSegments: 68, negativeSegments: 0
5864: positiveSegments: 7, negativeSegments: 3
5865: positiveSegments: 19, negativeSegments: 3
5866: positiveSegments: 20, negativeSegments: 9
5869: positiveSegments: 14, negativeSegments: 9
5870: positiveSegments: 5, negativeSegments: 9
5871: exit early, no segments to save
5871: positiveSegments: 0, negativeSegments: 0
5872: positiveSegments: 24, negativeSegments: 6
5873: positiveSegments: 8, negativeSegments: 3
5875: positiveSegments: 32, negativeSegments: 3
5882: exit early, no segments to save
5882: positiveSegments: 0, negativeSegments: 0
5884: positiveSegments: 8, negativeSegments: 0
5887: positiveSegments: 8, negativeSegments: 0
5888: positiveSegments: 4, negativeSegments: 3
5889: positiveSegments: 4, negativeSegments: 3
5890: positiveSegments: 12, negativeSegments: 3
5891: positiveSegments: 0, negativeSegments: 3
5892: positiveSegments: 4, negativeSegments: 3
5894: positiveSegments: 15, negativeSegments: 0
5895: positiveSegments: 5, negativeSegments: 9
5902: positiveSegments: 7, negativeSegments: 3
5904: positiveSegments: 16, negativeSegments: 6
5907: positiveSegments: 6, negativeSegments: 3
5908: exit early, no segments to save
5908: positiveSegments: 0, negativeSegments: 0
5911: positiveSegments: 8, negativeSegments: 3
5912: positiveSegments: 4, negativeSegments: 6
5914: positiveSegments: 12, negativeSegments: 3
5916: exit early, no segments to save
5916: positiveSegments: 0, negativeSegments: 0
5917: positiveSegments: 10, negativeSegments: 0
5918: positiveSegments: 4, negativeSegments: 9
5933: positiveSegments: 9, negativeSegments: 0
5934: positiveSegments: 1, negativeSegments: 6
5937: positiveSegments: 15, negativeSegments: 0
5938: positiveSegments: 4, negativeSegments: 6
5940: positiveSegments: 4, negativeSegments: 0
5942: positiveSegments: 11, negativeSegments: 3
5943: positiveSegments: 4, negativeSegments: 3
5944: positiveSegments: 12, negativeSegments: 9
5945: positiveSegments: 8, negativeSegments: 0
5946: positiveSegments: 12, negativeSegments: 6
5948: positiveSegments: 9, negativeSegments: 0
5950: positiveSegments: 29, negativeSegments: 9
5951: positiveSegments: 0, negativeSegments: 3
5954: positiveSegments: 7, negativeSegments: 0
5956: positiveSegments: 8, negativeSegments: 6
5958: positiveSegments: 0, negativeSegments: 15
5959: exit early, no segments to save
5959: positiveSegments: 0, negativeSegments: 0
5961: exit early, no segments to save
5961: positiveSegments: 0, negativeSegments: 0
5964: positiveSegments: 0, negativeSegments: 18
5965: positiveSegments: 19, negativeSegments: 0
5966: positiveSegments: 8, negativeSegments: 6
5967: positiveSegments: 21, negativeSegments: 9
5970: positiveSegments: 4, negativeSegments: 3
5971: nothing saved, all segments filtered
5971: positiveSegments: 1, negativeSegments: 0
5973: positiveSegments: 2, negativeSegments: 6
count processed: 2900, current case index: 5974
5974: exit early, no segments to save
5974: positiveSegments: 0, negativeSegments: 0
5975: positiveSegments: 12, negativeSegments: 3
5976: positiveSegments: 16, negativeSegments: 0
5977: positiveSegments: 15, negativeSegments: 9
5981: positiveSegments: 19, negativeSegments: 0
5982: positiveSegments: 6, negativeSegments: 0
5983: positiveSegments: 19, negativeSegments: 0
5986: positiveSegments: 8, negativeSegments: 9
5987: positiveSegments: 4, negativeSegments: 3
5989: positiveSegments: 13, negativeSegments: 3
5993: positiveSegments: 24, negativeSegments: 9
5994: positiveSegments: 7, negativeSegments: 0
5997: positiveSegments: 10, negativeSegments: 6
6000: positiveSegments: 4, negativeSegments: 21
6003: positiveSegments: 12, negativeSegments: 0
6006: nothing saved, all segments filtered
6006: positiveSegments: 3, negativeSegments: 0
6007: positiveSegments: 17, negativeSegments: 3
6009: positiveSegments: 36, negativeSegments: 6
6010: positiveSegments: 4, negativeSegments: 12
6013: positiveSegments: 8, negativeSegments: 0
6015: positiveSegments: 8, negativeSegments: 0
6016: positiveSegments: 0, negativeSegments: 3
6017: positiveSegments: 1, negativeSegments: 9
6020: positiveSegments: 5, negativeSegments: 15
6022: positiveSegments: 16, negativeSegments: 0
6027: positiveSegments: 4, negativeSegments: 0
6028: positiveSegments: 11, negativeSegments: 0
6029: positiveSegments: 2, negativeSegments: 3
6031: positiveSegments: 8, negativeSegments: 3
6032: exit early, no segments to save
6032: positiveSegments: 0, negativeSegments: 0
6037: positiveSegments: 12, negativeSegments: 6
6039: positiveSegments: 17, negativeSegments: 12
6041: exit early, no segments to save
6041: positiveSegments: 0, negativeSegments: 0
6042: positiveSegments: 7, negativeSegments: 0
6043: positiveSegments: 15, negativeSegments: 6
6047: positiveSegments: 3, negativeSegments: 9
6053: positiveSegments: 1, negativeSegments: 6
6055: positiveSegments: 6, negativeSegments: 0
6056: positiveSegments: 3, negativeSegments: 0
6057: exit early, no segments to save
6057: positiveSegments: 0, negativeSegments: 0
6058: positiveSegments: 32, negativeSegments: 9
6059: positiveSegments: 0, negativeSegments: 12
6060: positiveSegments: 0, negativeSegments: 12
6061: positiveSegments: 11, negativeSegments: 15
6063: positiveSegments: 11, negativeSegments: 9
6065: positiveSegments: 3, negativeSegments: 3
6066: positiveSegments: 18, negativeSegments: 0
6067: exit early, no segments to save
6067: positiveSegments: 0, negativeSegments: 0
6069: positiveSegments: 30, negativeSegments: 12
6070: positiveSegments: 0, negativeSegments: 6
6071: positiveSegments: 26, negativeSegments: 0
6074: positiveSegments: 4, negativeSegments: 18
6076: positiveSegments: 11, negativeSegments: 6
6077: positiveSegments: 17, negativeSegments: 6
6080: positiveSegments: 6, negativeSegments: 0
6082: positiveSegments: 29, negativeSegments: 3
6083: positiveSegments: 13, negativeSegments: 9
6084: positiveSegments: 44, negativeSegments: 18
6085: positiveSegments: 20, negativeSegments: 3
6086: positiveSegments: 21, negativeSegments: 6
6087: positiveSegments: 8, negativeSegments: 3
6088: exit early, no segments to save
6088: positiveSegments: 0, negativeSegments: 0
6089: positiveSegments: 0, negativeSegments: 6
6097: exit early, no segments to save
6097: positiveSegments: 0, negativeSegments: 0
6098: positiveSegments: 5, negativeSegments: 3
6101: positiveSegments: 6, negativeSegments: 6
6102: positiveSegments: 9, negativeSegments: 3
6103: positiveSegments: 0, negativeSegments: 12
6104: positiveSegments: 4, negativeSegments: 6
6109: exit early, no segments to save
6109: positiveSegments: 0, negativeSegments: 0
6114: positiveSegments: 26, negativeSegments: 0
6119: positiveSegments: 8, negativeSegments: 12
6121: positiveSegments: 25, negativeSegments: 3
6124: positiveSegments: 16, negativeSegments: 0
6126: positiveSegments: 2, negativeSegments: 18
6127: positiveSegments: 11, negativeSegments: 0
6129: positiveSegments: 14, negativeSegments: 12
6131: nothing saved, all segments filtered
6131: positiveSegments: 2, negativeSegments: 0
6132: positiveSegments: 13, negativeSegments: 0
6133: positiveSegments: 6, negativeSegments: 9
6134: positiveSegments: 3, negativeSegments: 9
6135: positiveSegments: 5, negativeSegments: 3
6136: positiveSegments: 8, negativeSegments: 6
6140: positiveSegments: 2, negativeSegments: 3
6141: positiveSegments: 15, negativeSegments: 0
6143: positiveSegments: 8, negativeSegments: 9
6144: positiveSegments: 7, negativeSegments: 0
6147: positiveSegments: 0, negativeSegments: 15
6152: positiveSegments: 15, negativeSegments: 0
6153: positiveSegments: 12, negativeSegments: 9
6154: positiveSegments: 4, negativeSegments: 0
6156: positiveSegments: 0, negativeSegments: 6
6159: positiveSegments: 43, negativeSegments: 3
6160: positiveSegments: 31, negativeSegments: 0
6163: positiveSegments: 16, negativeSegments: 3
6166: positiveSegments: 42, negativeSegments: 3
6167: positiveSegments: 0, negativeSegments: 15
6168: positiveSegments: 0, negativeSegments: 3
6169: positiveSegments: 0, negativeSegments: 12
6171: positiveSegments: 15, negativeSegments: 6
count processed: 3000, current case index: 6174
6174: positiveSegments: 0, negativeSegments: 6
6176: positiveSegments: 23, negativeSegments: 9
6178: positiveSegments: 23, negativeSegments: 3
6179: positiveSegments: 35, negativeSegments: 9
6180: nothing saved, all segments filtered
6180: positiveSegments: 3, negativeSegments: 0
6182: positiveSegments: 15, negativeSegments: 6
6184: positiveSegments: 3, negativeSegments: 12
6185: positiveSegments: 4, negativeSegments: 6
6186: positiveSegments: 0, negativeSegments: 6
6190: positiveSegments: 9, negativeSegments: 9
6191: positiveSegments: 8, negativeSegments: 3
6192: exit early, no segments to save
6192: positiveSegments: 0, negativeSegments: 0
6194: positiveSegments: 2, negativeSegments: 3
6195: positiveSegments: 16, negativeSegments: 3
6196: positiveSegments: 0, negativeSegments: 9
6198: positiveSegments: 3, negativeSegments: 6
6199: positiveSegments: 7, negativeSegments: 6
6200: positiveSegments: 7, negativeSegments: 3
6204: positiveSegments: 1, negativeSegments: 0
6205: exit early, no segments to save
6205: positiveSegments: 0, negativeSegments: 0
6206: positiveSegments: 6, negativeSegments: 0
6208: positiveSegments: 0, negativeSegments: 3
6210: positiveSegments: 1, negativeSegments: 3
6214: positiveSegments: 13, negativeSegments: 6
6217: positiveSegments: 37, negativeSegments: 3
6218: positiveSegments: 7, negativeSegments: 6
6219: positiveSegments: 7, negativeSegments: 9
6220: positiveSegments: 30, negativeSegments: 12
6224: positiveSegments: 4, negativeSegments: 6
6227: positiveSegments: 21, negativeSegments: 0
6228: positiveSegments: 9, negativeSegments: 0
6230: positiveSegments: 8, negativeSegments: 3
6233: positiveSegments: 6, negativeSegments: 0
6235: positiveSegments: 0, negativeSegments: 3
6238: positiveSegments: 14, negativeSegments: 3
6239: positiveSegments: 19, negativeSegments: 6
6240: positiveSegments: 4, negativeSegments: 0
6241: positiveSegments: 7, negativeSegments: 9
6248: positiveSegments: 19, negativeSegments: 6
6250: positiveSegments: 4, negativeSegments: 6
6254: positiveSegments: 5, negativeSegments: 6
6255: positiveSegments: 11, negativeSegments: 12
6257: positiveSegments: 5, negativeSegments: 24
6260: positiveSegments: 10, negativeSegments: 0
6261: exit early, no segments to save
6261: positiveSegments: 0, negativeSegments: 0
6262: positiveSegments: 5, negativeSegments: 3
6264: positiveSegments: 0, negativeSegments: 15
6266: positiveSegments: 21, negativeSegments: 3
6267: positiveSegments: 4, negativeSegments: 6
6268: positiveSegments: 4, negativeSegments: 3
6269: positiveSegments: 13, negativeSegments: 0
6270: positiveSegments: 5, negativeSegments: 18
6271: positiveSegments: 1, negativeSegments: 0
6273: positiveSegments: 11, negativeSegments: 6
6275: positiveSegments: 4, negativeSegments: 0
6277: positiveSegments: 3, negativeSegments: 0
6279: positiveSegments: 5, negativeSegments: 12
6280: positiveSegments: 2, negativeSegments: 9
6281: positiveSegments: 26, negativeSegments: 0
6282: positiveSegments: 16, negativeSegments: 6
6284: positiveSegments: 9, negativeSegments: 3
6286: positiveSegments: 1, negativeSegments: 3
6289: positiveSegments: 6, negativeSegments: 3
6290: positiveSegments: 21, negativeSegments: 0
6292: positiveSegments: 10, negativeSegments: 3
6293: exit early, no segments to save
6293: positiveSegments: 0, negativeSegments: 0
6295: positiveSegments: 8, negativeSegments: 6
6296: positiveSegments: 19, negativeSegments: 6
6297: positiveSegments: 35, negativeSegments: 0
6298: positiveSegments: 5, negativeSegments: 6
6302: positiveSegments: 10, negativeSegments: 0
6305: positiveSegments: 3, negativeSegments: 12
6306: positiveSegments: 6, negativeSegments: 3
6307: positiveSegments: 12, negativeSegments: 3
6309: positiveSegments: 68, negativeSegments: 3
6311: positiveSegments: 16, negativeSegments: 3
6312: positiveSegments: 14, negativeSegments: 0
6314: positiveSegments: 18, negativeSegments: 3
6315: positiveSegments: 4, negativeSegments: 0
6316: positiveSegments: 17, negativeSegments: 3
6317: positiveSegments: 23, negativeSegments: 0
6324: positiveSegments: 5, negativeSegments: 0
6330: positiveSegments: 4, negativeSegments: 0
6331: positiveSegments: 3, negativeSegments: 0
6332: positiveSegments: 8, negativeSegments: 15
6339: positiveSegments: 4, negativeSegments: 0
6343: positiveSegments: 8, negativeSegments: 12
6345: positiveSegments: 8, negativeSegments: 9
6346: positiveSegments: 0, negativeSegments: 6
6351: positiveSegments: 20, negativeSegments: 9
6355: positiveSegments: 3, negativeSegments: 6
6357: positiveSegments: 8, negativeSegments: 12
6359: positiveSegments: 7, negativeSegments: 0
6360: positiveSegments: 30, negativeSegments: 0
6361: positiveSegments: 26, negativeSegments: 3
6362: positiveSegments: 15, negativeSegments: 3
6363: positiveSegments: 12, negativeSegments: 12
6366: positiveSegments: 2, negativeSegments: 6
6368: positiveSegments: 8, negativeSegments: 0
6370: positiveSegments: 0, negativeSegments: 6
count processed: 3100, current case index: 6372
6372: positiveSegments: 0, negativeSegments: 9
6373: positiveSegments: 2, negativeSegments: 0
6375: positiveSegments: 10, negativeSegments: 0
6376: positiveSegments: 12, negativeSegments: 0
6378: positiveSegments: 18, negativeSegments: 0
6381: positiveSegments: 0, negativeSegments: 3
6383: positiveSegments: 4, negativeSegments: 15
6385: positiveSegments: 29, negativeSegments: 0
6386: positiveSegments: 8, negativeSegments: 15
6388: positiveSegments: 9, negativeSegments: 0
extracted: 3110

Track and Segment Validity Checks

def printAbp(case_id_to_check, plot_invalid_only=False):
        vf_path = f'{VITAL_MINI}/{case_id_to_check:04d}_mini.vital'
        
        if not os.path.isfile(vf_path):
              return
        
        vf = vitaldb.VitalFile(vf_path)
        abp = vf.to_numpy(TRACK_NAMES[0], 1/500)
        
        print(f'Case {case_id_to_check}')
        print(f'ABP Shape: {abp.shape}')

        print(f'nanmin: {np.nanmin(abp)}')
        print(f'nanmean: {np.nanmean(abp)}')
        print(f'nanmax: {np.nanmax(abp)}')
        
        is_valid = isAbpSegmentValidNumpy(abp, debug=True)
        print(f'valid: {is_valid}')

        if plot_invalid_only and is_valid:
            return
            
        plt.figure(figsize=(20, 5))
        plt_color = 'C0' if is_valid else 'red'
        plt.plot(abp, plt_color)
        plt.title(f'ABP - Entire Track - Case {case_id_to_check} - {abp.shape[0] / 500} seconds')
        plt.axhline(y = 65, color = 'maroon', linestyle = '--')
        plt.show()

def printSegments(segmentsMap, case_id_to_check, print_label, normalize=False):
    for (x1, x2, r, abp, ecg, eeg) in segmentsMap[case_id_to_check]:
        print(f'{print_label}: Case {case_id_to_check}')
        print(f'lookback window: {r} min')
        print(f'start time: {x1}')
        print(f'end time: {x2}')
        print(f'length: {x2 - x1} sec')
        
        print(f'ABP Shape: {abp.shape}')
        print(f'ECG Shape: {ecg.shape}')
        print(f'EEG Shape: {eeg.shape}')

        print(f'nanmin: {np.nanmin(abp)}')
        print(f'nanmean: {np.nanmean(abp)}')
        print(f'nanmax: {np.nanmax(abp)}')
        
        is_valid = isAbpSegmentValidNumpy(abp, debug=True)
        print(f'valid: {is_valid}')

        # ABP normalization
        x_abp = np.copy(abp)
        if normalize:
            x_abp -= 65
            x_abp /= 65

        plt.figure(figsize=(20, 5))
        plt_color = 'C0' if is_valid else 'red'
        plt.plot(x_abp, plt_color)
        plt.title('ABP')
        plt.axhline(y = 65, color = 'maroon', linestyle = '--')
        plt.show()

        plt.figure(figsize=(20, 5))
        plt.plot(ecg, 'teal')
        plt.title('ECG')
        plt.show()

        plt.figure(figsize=(20, 5))
        plt.plot(eeg, 'indigo')
        plt.title('EEG')
        plt.show()

        print()

def printEvents(abp_raw, eventsMap, case_id_to_check, print_label, normalize=False):
    for (x1, x2) in eventsMap[case_id_to_check]:
        print(f'{print_label}: Case {case_id_to_check}')
        print(f'start time: {x1}')
        print(f'end time: {x2}')
        print(f'length: {x2 - x1} sec')

        abp = abp_raw[x1*500:x2*500]
        print(f'ABP Shape: {abp.shape}')

        print(f'nanmin: {np.nanmin(abp)}')
        print(f'nanmean: {np.nanmean(abp)}')
        print(f'nanmax: {np.nanmax(abp)}')
        
        is_valid = isAbpSegmentValidNumpy(abp, debug=True)
        print(f'valid: {is_valid}')

        # ABP normalization
        x_abp = np.copy(abp)
        if normalize:
            x_abp -= 65
            x_abp /= 65

        plt.figure(figsize=(20, 5))
        plt_color = 'C0' if is_valid else 'red'
        plt.plot(x_abp, plt_color)
        plt.title('ABP')
        plt.axhline(y = 65, color = 'maroon', linestyle = '--')
        plt.show()

        print()

def moving_average(x, seconds=60):
    w = seconds * 500
    return np.convolve(np.squeeze(x), np.ones(w), 'valid') / w

def printAbpOverlay(
    case_id_to_check,
    positiveSegmentsMap,
    negativeSegmentsMap,
    iohEventsMap,
    cleanEventsMap,
    movingAverage=False
):
    def overlay_segments(plt, segmentsMap, color, linestyle, positive=False):
        for (x1, x2, r, abp, ecg, eeg) in segmentsMap:
            sx1 = x1*500
            sx2 = x2*500
            mycolor = color
            if positive:
                if r == 3:
                    mycolor = 'red'
                elif r == 5:
                    mycolor = 'crimson'
                elif r == 10:
                    mycolor = 'tomato'
                else:
                    mycolor = 'salmon'
            plt.axvline(x = sx1, color = mycolor, linestyle = linestyle)
            plt.axvline(x = sx2, color = mycolor, linestyle = linestyle)
            plt.axvspan(sx1, sx2, facecolor = mycolor, alpha = 0.1)

    def overlay_events(plt, eventsMap, color, linestyle):
        for (x1, x2) in eventsMap:
            sx1 = x1*500
            sx2 = x2*500
            plt.axvline(x = sx1, color = color, linestyle = linestyle)
            plt.axvline(x = sx2, color = color, linestyle = linestyle)
            plt.axvspan(sx1, sx2, facecolor = color, alpha = 0.1)
    
    vf_path = f'{VITAL_MINI}/{case_id_to_check:04d}_mini.vital'

    if not os.path.isfile(vf_path):
          return

    vf = vitaldb.VitalFile(vf_path)
    abp = vf.to_numpy(TRACK_NAMES[0], 1/500)
    
    abp_mov_avg = None
    if movingAverage:
        abp_mov_avg = moving_average(abp)

    print(f'Case {case_id_to_check}')
    print(f'ABP Shape: {abp.shape}')

    print(f'nanmin: {np.nanmin(abp)}')
    print(f'nanmean: {np.nanmean(abp)}')
    print(f'nanmax: {np.nanmax(abp)}')

    is_valid = isAbpSegmentValidNumpy(abp, debug=True)
    print(f'valid: {is_valid}')

    plt.figure(figsize=(24, 8))
    plt_color = 'C0' if is_valid else 'red'
    plt.plot(abp, plt_color)
    plt.title(f'ABP - Entire Track - Case {case_id_to_check} - {abp.shape[0] / 500} seconds')
    plt.axhline(y = 65, color = 'maroon', linestyle = '--')
    
    if movingAverage:
        plt.plot(abp_mov_avg, 'maroon')

    # https://matplotlib.org/stable/gallery/lines_bars_and_markers/linestyles.html#linestyles
    
    overlay_segments(plt, positiveSegmentsMap[case_id_to_check], 'crimson', (0, (1, 1)), positive=True)
    
    overlay_segments(plt, negativeSegmentsMap[case_id_to_check], 'teal', (0, (1, 1)))

    overlay_events(plt, iohEventsMap[case_id_to_check], 'brown', '-')
    
    overlay_events(plt, cleanEventsMap[case_id_to_check], 'teal', '-')

    plt.show()

Reality Check All Cases

# Check if all ABPs are well formed.
DISPLAY_REALITY_CHECK_ABP=True
DISPLAY_REALITY_CHECK_ABP_FIRST_ONLY=True

if DISPLAY_REALITY_CHECK_ABP:
    for case_id_to_check in cases_of_interest_idx:
        printAbp(case_id_to_check, plot_invalid_only=False)
        
        if DISPLAY_REALITY_CHECK_ABP_FIRST_ONLY:
            break

Case 1
ABP Shape: (5770575, 1)
nanmin: -495.6260070800781
nanmean: 78.15251159667969
nanmax: 374.3236389160156
Presence of BP > 200
valid: False

Validate Malformed Vital Files - Missing One Or More Tracks

# These are Vital Files removed because of malformed ABP waveforms.
DISPLAY_MALFORMED_ABP=True
DISPLAY_MALFORMED_ABP_FIRST_ONLY=True

if DISPLAY_MALFORMED_ABP:
    malformed_case_ids = pd.read_csv('malformed_tracks_filter.csv', header=None, names=['caseid']).set_index('caseid').index

    for case_id_to_check in malformed_case_ids:
        printAbp(case_id_to_check)
        
        if DISPLAY_MALFORMED_ABP_FIRST_ONLY:
            break

Case 3
ABP Shape: (2196524, 1)
nanmin: -117.43000030517578
nanmean: 0.6060270667076111
nanmax: 85.98619842529297
Presence of BP < 30
valid: False

Validate Cases With No Segments Saved

DISPLAY_NO_SEGMENTS_CASES=True
DISPLAY_NO_SEGMENTS_CASES_FIRST_ONLY=True

if DISPLAY_NO_SEGMENTS_CASES:
    no_segments_case_ids = [3413, 3476, 3533, 3992, 4328, 4648, 4703, 4733, 5130, 5501, 5693, 5908]

    for case_id_to_check in no_segments_case_ids:
        printAbp(case_id_to_check)
        
        if DISPLAY_NO_SEGMENTS_CASES_FIRST_ONLY:
            break

Case 3413
ABP Shape: (3429927, 1)
nanmin: -228.025146484375
nanmean: 48.4425163269043
nanmax: 293.3521423339844
>10% NaN
valid: False

Select Case For Segment Extraction Validation

Generate segment data for one or more cases. Perform a deep analysis of event and segment quality.

#mycoi = cases_of_interest_idx
my_cases_of_interest_idx = cases_of_interest_idx[:10]
#mycoi = [1]

positiveSegmentsMap, negativeSegmentsMap, iohEventsMap, cleanEventsMap = \
    extract_segments(my_cases_of_interest_idx, debug=False, checkCache=False, 
                     forceWrite=False, returnSegments=True, skipInvalidCleanEvents=True)

1: positiveSegments: 12, negativeSegments: 3
4: positiveSegments: 22, negativeSegments: 3
7: positiveSegments: 12, negativeSegments: 6
10: positiveSegments: 27, negativeSegments: 6
12: positiveSegments: 22, negativeSegments: 0
13: positiveSegments: 18, negativeSegments: 0
16: positiveSegments: 12, negativeSegments: 6
19: positiveSegments: 34, negativeSegments: 3
20: positiveSegments: 17, negativeSegments: 9
22: positiveSegments: 16, negativeSegments: 12

Select a specific case to perform detailed low level analysis.

case_id_to_check = my_cases_of_interest_idx[0]
case_id_to_check

1

print((
    len(positiveSegmentsMap[case_id_to_check]),
    len(negativeSegmentsMap[case_id_to_check]),
    len(iohEventsMap[case_id_to_check]),
    len(cleanEventsMap[case_id_to_check])
))

(12, 3, 7, 3)

printAbp(case_id_to_check)

Case 1
ABP Shape: (5770575, 1)
nanmin: -495.6260070800781
nanmean: 78.15251159667969
nanmax: 374.3236389160156
Presence of BP > 200
valid: False

Positive Events for Case - IOH Events

Used to define the range in front of which positive segments will be extracted. Positive samples happen in front of this region.

tmp_vf_path = f'{VITAL_MINI}/{case_id_to_check:04d}_mini.vital'
tmp_vf = vitaldb.VitalFile(tmp_vf_path)
tmp_abp = tmp_vf.to_numpy(TRACK_NAMES[0], 1/500)

printEvents(tmp_abp, iohEventsMap, case_id_to_check, 'IOH Event Segment', normalize=False)

IOH Event Segment: Case 1
start time: 1788
end time: 1849
length: 61 sec
ABP Shape: (30500, 1)
nanmin: 32.663482666015625
nanmean: 64.93988037109375
nanmax: 123.50955200195312
valid: True

IOH Event Segment: Case 1
start time: 1850
end time: 2113
length: 263 sec
ABP Shape: (131500, 1)
nanmin: 37.600799560546875
nanmean: 63.139060974121094
nanmax: 101.78549194335938
valid: True

IOH Event Segment: Case 1
start time: 2314
end time: 2375
length: 61 sec
ABP Shape: (30500, 1)
nanmin: -262.5861511230469
nanmean: 65.14369201660156
nanmax: 343.7124938964844
Presence of BP > 200
valid: False

IOH Event Segment: Case 1
start time: 4113
end time: 4199
length: 86 sec
ABP Shape: (43000, 1)
nanmin: 22.788909912109375
nanmean: 65.0725326538086
nanmax: 153.13327026367188
Presence of BP < 30
valid: False

IOH Event Segment: Case 1
start time: 4261
end time: 5350
length: 1089 sec
ABP Shape: (544500, 1)
nanmin: 36.613311767578125
nanmean: 60.451026916503906
nanmax: 110.67263793945312
valid: True

IOH Event Segment: Case 1
start time: 9096
end time: 9156
length: 60 sec
ABP Shape: (30000, 1)
nanmin: 40.563140869140625
nanmean: 64.9837646484375
nanmax: 108.69772338867188
valid: True

IOH Event Segment: Case 1
start time: 9157
end time: 9503
length: 346 sec
ABP Shape: (173000, 1)
nanmin: 39.575714111328125
nanmean: 62.33021545410156
nanmax: 104.74789428710938
valid: True

Negative Events for Case - Non-IOH Events

Used to define the range from in which negative segments will be extracted. Negative samples happen within this region.

printEvents(tmp_abp, cleanEventsMap, case_id_to_check, 'Clean Event Segment', normalize=False)

Clean Event Segment: Case 1
start time: 5351
end time: 7151
length: 1800 sec
ABP Shape: (900000, 1)
nanmin: 40.563140869140625
nanmean: 84.04818725585938
nanmax: 151.15835571289062
valid: True

Clean Event Segment: Case 1
start time: 7152
end time: 8952
length: 1800 sec
ABP Shape: (900000, 1)
nanmin: -495.6260070800781
nanmean: 99.71124267578125
nanmax: 368.3988952636719
Presence of BP > 200
valid: False

Clean Event Segment: Case 1
start time: 9504
end time: 11304
length: 1800 sec
ABP Shape: (900000, 1)
nanmin: -49.295440673828125
nanmean: 83.3201675415039
nanmax: 346.6748352050781
Presence of BP > 200
valid: False

Positive Segments for Case - IOH Events Predicted Using These

One minute regions sampled and used for training the model for "positive" events.

printSegments(positiveSegmentsMap, case_id_to_check, 'Positive Segment - IOH Event', normalize=False)

Positive Segment - IOH Event: Case 1
lookback window: 3 min
start time: 1548
end time: 1608
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 46.487884521484375
nanmean: 73.00869750976562
nanmax: 113.63497924804688
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 5 min
start time: 1428
end time: 1488
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 41.550628662109375
nanmean: 74.47395324707031
nanmax: 128.44686889648438
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 10 min
start time: 1128
end time: 1188
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 53.400115966796875
nanmean: 88.63211059570312
nanmax: 135.35903930664062
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 15 min
start time: 828
end time: 888
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 23.776397705078125
nanmean: 108.88127136230469
nanmax: 182.75698852539062
Presence of BP < 30
valid: False

Positive Segment - IOH Event: Case 1
lookback window: 3 min
start time: 3873
end time: 3933
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 46.487884521484375
nanmean: 75.3544692993164
nanmax: 124.49703979492188
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 5 min
start time: 3753
end time: 3813
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 45.500457763671875
nanmean: 73.97709655761719
nanmax: 122.52212524414062
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 10 min
start time: 3453
end time: 3513
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 52.412628173828125
nanmean: 86.52787780761719
nanmax: 148.19595336914062
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 15 min
start time: 3153
end time: 3213
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 58.337371826171875
nanmean: 100.94121551513672
nanmax: 165.97018432617188
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 3 min
start time: 8856
end time: 8916
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 64.26211547851562
nanmean: 97.06536102294922
nanmax: 157.08309936523438
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 5 min
start time: 8736
end time: 8796
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 69.19943237304688
nanmean: 105.55238342285156
nanmax: 163.00784301757812
valid: True

Positive Segment - IOH Event: Case 1
lookback window: 10 min
start time: 8436
end time: 8496
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: -88.793701171875
nanmean: 130.62982177734375
nanmax: 305.2016296386719
Presence of BP > 200
valid: False

Positive Segment - IOH Event: Case 1
lookback window: 15 min
start time: 8136
end time: 8196
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 62.287200927734375
nanmean: 92.04357147216797
nanmax: 138.32138061523438
valid: True

Negative Segments for Case - Non-IOH Events Predicted Using These

One minute regions sampled and used for training the model for "negative" events.

printSegments(negativeSegmentsMap, case_id_to_check, 'Negative Segment - Non-Event', normalize=False)

Negative Segment - Non-Event: Case 1
lookback window: 0 min
start time: 5951
end time: 6011
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 52.412628173828125
nanmean: 76.35643005371094
nanmax: 120.54721069335938
valid: True

Negative Segment - Non-Event: Case 1
lookback window: 0 min
start time: 6251
end time: 6311
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 54.387542724609375
nanmean: 77.73150634765625
nanmax: 120.54721069335938
valid: True

Negative Segment - Non-Event: Case 1
lookback window: 0 min
start time: 6551
end time: 6611
length: 60 sec
ABP Shape: (30000,)
ECG Shape: (30000,)
EEG Shape: (7680,)
nanmin: 58.337371826171875
nanmean: 85.06976318359375
nanmax: 133.38412475585938
valid: True

Overlay Plot of All Events and Segments Extracted

For each of the cases in my_cases_of_interest_idx overlay the results of event and segment extraction.

DISPLAY_OVERLAY_CHECK_ABP=True
DISPLAY_OVERLAY_CHECK_ABP_FIRST_ONLY=False

if DISPLAY_REALITY_CHECK_ABP:
    for case_id_to_check in my_cases_of_interest_idx:
        printAbpOverlay(case_id_to_check, positiveSegmentsMap, 
                        negativeSegmentsMap, iohEventsMap, cleanEventsMap, movingAverage=True)
        
        if DISPLAY_OVERLAY_CHECK_ABP_FIRST_ONLY:
            break

Case 1
ABP Shape: (5770575, 1)
nanmin: -495.6260070800781
nanmean: 78.15251159667969
nanmax: 374.3236389160156
Presence of BP > 200
valid: False

Case 4
ABP Shape: (10494700, 1)
nanmin: -495.6260070800781
nanmean: 70.51863098144531
nanmax: 356.5494079589844
Presence of BP > 200
valid: False

Case 7
ABP Shape: (7884623, 1)
nanmin: -495.6260070800781
nanmean: 73.2461166381836
nanmax: 342.7250061035156
Presence of BP > 200
valid: False

Case 10
ABP Shape: (10494487, 1)
nanmin: -495.6260070800781
nanmean: 81.80549621582031
nanmax: 312.1138610839844
Presence of BP > 200
valid: False

Case 12
ABP Shape: (15601433, 1)
nanmin: -495.6260070800781
nanmean: 71.71157836914062
nanmax: 399.0100402832031
>10% NaN
valid: False

Case 13
ABP Shape: (5405175, 1)
nanmin: -495.6260070800781
nanmean: 59.84370422363281
nanmax: 223.24270629882812
Presence of BP > 200
valid: False

Case 16
ABP Shape: (6432446, 1)
nanmin: -495.6260070800781
nanmean: 79.41276550292969
nanmax: 406.9096984863281
Presence of BP > 200
valid: False

Case 19
ABP Shape: (13785944, 1)
nanmin: -495.6260070800781
nanmean: 69.16629791259766
nanmax: 407.8971862792969
Presence of BP > 200
valid: False

Case 20
ABP Shape: (13237618, 1)
nanmin: -495.6260070800781
nanmean: 75.18829345703125
nanmax: 305.2016296386719
Presence of BP > 200
valid: False

Case 22
ABP Shape: (7186505, 1)
nanmin: -410.7047119140625
nanmean: 87.67796325683594
nanmax: 407.8971862792969
Presence of BP > 200
valid: False

# free memory
del tmp_abp

Generate Train/Val/Test Splits

def get_segment_attributes_from_filename(file_path):
    pieces = os.path.basename(file_path).split('_')
    case = int(pieces[0])
    startX = int(pieces[1])
    predWindow = int(pieces[2])
    label = pieces[3].replace('.h5', '')
    return (case, startX, predWindow, label)

count_negative_samples = 0
count_positive_samples = 0

samples = []

from glob import glob
seg_folder = f"{VITAL_EXTRACTED_SEGMENTS}"
filenames = [y for x in os.walk(seg_folder) for y in glob(os.path.join(x[0], '*.h5'))]

for filename in filenames:
    (case, start_x, pred_window, label) = get_segment_attributes_from_filename(filename)
    #print((case, start_x, pred_window, label))
    
    # only load cases for cases of interest; this folder could have segments for hundreds of cases
    if case not in cases_of_interest_idx:
        continue

    #PREDICTION_WINDOW = 3
    if pred_window == 0 or pred_window == PREDICTION_WINDOW or PREDICTION_WINDOW == 'ALL':
        #print((case, start_x, pred_window, label))
        if label == 'True':
            count_positive_samples += 1
        else:
            count_negative_samples += 1
        sample = (filename, label)
        samples.append(sample)

print()
print(f"samples loaded:         {len(samples):5} ")
print(f'count negative samples: {count_negative_samples:5}')
print(f'count positive samples: {count_positive_samples:5}')

samples loaded:         22277 
count negative samples: 14125
count positive samples:  8152

# Divide by cases
sample_cases = defaultdict(lambda: []) 

for fn, _ in samples:
    (case, start_x, pred_window, label) = get_segment_attributes_from_filename(fn)
    sample_cases[case].append((fn, label))

# understand any missing cases of interest
sample_cases_idx = pd.Index(sample_cases.keys())
missing_case_ids = cases_of_interest_idx.difference(sample_cases_idx)
print(f'cases with no samples: {missing_case_ids.shape[0]}')
print(f'    {missing_case_ids}')
print()
    
# Split data into training, validation, and test sets
# Use 6:1:3 ratio and prevent samples from a single case from being split across different sets
# Note: number of samples at each time point is not the same, because the first event can occur before the 3/5/10/15 minute mark

# Set target sizes
train_ratio = 0.6
val_ratio = 0.1
test_ratio = 1 - train_ratio - val_ratio # ensure ratios sum to 1

# Split samples into train and other
sample_cases_train, sample_cases_other = train_test_split(list(sample_cases.keys()), test_size=(1 - train_ratio), random_state=RANDOM_SEED)

# Split other into val and test
sample_cases_val, sample_cases_test = train_test_split(sample_cases_other, test_size=(test_ratio / (1 - train_ratio)), random_state=RANDOM_SEED)

# Check how many samples are in each set
print(f'Train/Val/Test Summary by Cases')
print(f"Train cases:  {len(sample_cases_train):5}, ({len(sample_cases_train) / len(sample_cases):.2%})")
print(f"Val cases:    {len(sample_cases_val):5}, ({len(sample_cases_val) / len(sample_cases):.2%})")
print(f"Test cases:   {len(sample_cases_test):5}, ({len(sample_cases_test) / len(sample_cases):.2%})")
print(f"Total cases:  {(len(sample_cases_train) + len(sample_cases_val) + len(sample_cases_test)):5}")

cases with no samples: 251
    Index([  26,   69,   70,  126,  139,  172,  199,  218,  221,  258,
       ...
       6109, 6131, 6180, 6192, 6205, 6261, 6275, 6293, 6330, 6339],
      dtype='int64', length=251)

Train/Val/Test Summary by Cases
Train cases:   1715, (59.99%)
Val cases:      285, (9.97%)
Test cases:     859, (30.05%)
Total cases:   2859

sample_cases_train = set(sample_cases_train)
sample_cases_val = set(sample_cases_val)
sample_cases_test = set(sample_cases_test)

samples_train = []
samples_val = []
samples_test = []

for cid, segs in sample_cases.items():
    if cid in sample_cases_train:
        for seg in segs:
            samples_train.append(seg)
    if cid in sample_cases_val:
        for seg in segs:
            samples_val.append(seg)
    if cid in sample_cases_test:
        for seg in segs:
            samples_test.append(seg)
            
# Check how many samples are in each set
print(f'Train/Val/Test Summary by Events')
print(f"Train events:  {len(samples_train):5}, ({len(samples_train) / len(samples):.2%})")
print(f"Val events:    {len(samples_val):5}, ({len(samples_val) / len(samples):.2%})")
print(f"Test events:   {len(samples_test):5}, ({len(samples_test) / len(samples):.2%})")
print(f"Total events:  {(len(samples_train) + len(samples_val) + len(samples_test)):5}")

Train/Val/Test Summary by Events
Train events:  13337, (59.87%)
Val events:     2187, (9.82%)
Test events:    6753, (30.31%)
Total events:  22277

Validate train/val/test Splits

PRINT_ALL_CASE_SPLIT_DETAILS = False

case_to_sample_distribution = defaultdict(lambda: {'train': [0, 0], 'val': [0, 0], 'test': [0, 0]})

def populate_case_to_sample_distribution(mysamples, idx):
    neg = 0
    pos = 0
    
    for fn, _ in mysamples:
        (case, start_x, pred_window, label) = get_segment_attributes_from_filename(fn)
        slot = 0 if label == 'False' else 1
        case_to_sample_distribution[case][idx][slot] += 1
        if slot == 0:
            neg += 1
        else:
            pos += 1
                
    return (neg, pos)

train_neg, train_pos = populate_case_to_sample_distribution(samples_train, 'train')
val_neg, val_pos     = populate_case_to_sample_distribution(samples_val,   'val')
test_neg, test_pos   = populate_case_to_sample_distribution(samples_test,  'test')

print(f'Total Cases Present: {len(case_to_sample_distribution):5}')
print()

train_tot = train_pos + train_neg
val_tot = val_pos + val_neg
test_tot = test_pos + test_neg
print(f'Train: P: {train_pos:5} ({(train_pos/train_tot):.2}), N: {train_neg:5} ({(train_neg/train_tot):.2})')
print(f'Val:   P: {val_pos:5} ({(val_pos/val_tot):.2}), N: {val_neg:5} ({(val_neg/val_tot):.2})')
print(f'Test:  P: {test_pos:5} ({(test_pos/test_tot):.2}), N: {test_neg:5}  ({(test_neg/test_tot):.2})')
print()

total_pos = train_pos + val_pos + test_pos
total_neg = train_neg + val_neg + test_neg
total = total_pos + total_neg
print(f'P/N Ratio: {(total_pos)}:{(total_neg)}')
print(f'P Percent: {(total_pos/total):.2}')
print(f'N Percent: {(total_neg/total):.2}')
print()

if PRINT_ALL_CASE_SPLIT_DETAILS:
    for ci in sorted(case_to_sample_distribution.keys()):
        print(f'{ci}: {case_to_sample_distribution[ci]}')

Total Cases Present:  2859

Train: P:  4916 (0.37), N:  8421 (0.63)
Val:   P:   775 (0.35), N:  1412 (0.65)
Test:  P:  2461 (0.36), N:  4292  (0.64)

P/N Ratio: 8152:14125
P Percent: 0.37
N Percent: 0.63

def check_data_leakage(full_data, train_data, val_data, test_data):
    # Convert to sets for easier operations
    full_data_set = set(full_data)
    train_data_set = set(train_data)
    val_data_set = set(val_data)
    test_data_set = set(test_data)

    # Check if train, val, test are subsets of full_data
    if not train_data_set.issubset(full_data_set):
        return "Train data has leakage"
    if not val_data_set.issubset(full_data_set):
        return "Validation data has leakage"
    if not test_data_set.issubset(full_data_set):
        return "Test data has leakage"

    # Check if train, val, test are disjoint
    if train_data_set & val_data_set:
        return "Train and validation data are not disjoint"
    if train_data_set & test_data_set:
        return "Train and test data are not disjoint"
    if val_data_set & test_data_set:
        return "Validation and test data are not disjoint"

    return "No data leakage detected"

# Usage
print(check_data_leakage(list(sample_cases.keys()), sample_cases_train, sample_cases_val, sample_cases_test))

No data leakage detected

# Create vitalDataset class
class vitalDataset(Dataset):
    def __init__(self, samples, normalize_abp=False):
        self.samples = samples
        self.normalize_abp = normalize_abp

    def __len__(self):
        return len(self.samples)

    def __getitem__(self, idx):
        # Get metadata for this event
        segment = self.samples[idx]

        file_path = segment[0]
        label = (segment[1] == "True" or segment[1] == "True.vital")

        (abp, ecg, eeg) = get_segment_data(file_path)

        if abp is None or eeg is None or ecg is None:
            return (np.zeros(30000), np.zeros(30000), np.zeros(7680), 0)
        
        if self.normalize_abp:
            abp -= 65
            abp /= 65

        return abp, ecg, eeg, label

NORMALIZE_ABP = False

train_dataset = vitalDataset(samples_train, NORMALIZE_ABP)
val_dataset = vitalDataset(samples_val, NORMALIZE_ABP)
test_dataset = vitalDataset(samples_test, NORMALIZE_ABP)

train/val/test Splits Summary Statistics

def generate_nan_means(mydataset):
    xs = np.zeros(len(mydataset))
    ys = np.zeros(len(mydataset), dtype=int)

    for i, (abp, ecg, eeg, y) in enumerate(iter(mydataset)):
        xs[i] = np.nanmean(abp)
        ys[i] = int(y)

    return pd.DataFrame({'abp_nanmean': xs, 'label': ys})

def generate_nan_means_summaries(tr, va, te, group='all'):
    if group == 'all':
        return pd.DataFrame({
            'train': tr.describe()['abp_nanmean'],
            'validation': va.describe()['abp_nanmean'],
            'test': te.describe()['abp_nanmean']
        })
    
    mytr = tr.reset_index()
    myva = va.reset_index()
    myte = te.reset_index()
    
    label_flag = True if group == 'positive' else False
    
    return pd.DataFrame({
        'train':      mytr[mytr['label'] == label_flag].describe()['abp_nanmean'],
        'validation': myva[myva['label'] == label_flag].describe()['abp_nanmean'],
        'test':       myte[myte['label'] == label_flag].describe()['abp_nanmean']
    })

def plot_nan_means(df, plot_label):
    mydf = df.reset_index()

    maxCases = 'ALL' if MAX_CASES is None else MAX_CASES
    plot_title = f'{plot_label} - ABP nanmean Values, {PREDICTION_WINDOW} Minutes, {maxCases} Cases'
    
    ax = mydf[mydf['label'] == False].plot.scatter(
        x='index', y='abp_nanmean', color='DarkBlue', label='Negative', 
        title=plot_title, figsize=(16,9))

    negative_median = mydf[mydf['label'] == False]['abp_nanmean'].median()
    ax.axhline(y=negative_median, color='DarkBlue', linestyle='--', label='Negative Median')
    
    mydf[mydf['label'] == True].plot.scatter(
        x='index', y='abp_nanmean', color='DarkOrange', label='Positive', ax=ax);
    
    positive_median = mydf[mydf['label'] == True]['abp_nanmean'].median()
    ax.axhline(y=positive_median, color='DarkOrange', linestyle='--', label='Positive Median')
    
    ax.legend(loc='upper right')

def plot_nan_means_hist(df):
    df.plot.hist(column=['abp_nanmean'], by='label', bins=50, figsize=(10, 8));

train_abp_nanmeans = generate_nan_means(train_dataset)
val_abp_nanmeans = generate_nan_means(val_dataset)
test_abp_nanmeans = generate_nan_means(test_dataset)

ABP Nanmean Summaries

generate_nan_means_summaries(train_abp_nanmeans, val_abp_nanmeans, test_abp_nanmeans)

	train	validation	test
count	13337.000000	2187.000000	6753.000000
mean	84.516606	84.336261	84.230145
std	12.166553	12.211263	11.984190
min	65.093372	65.146761	65.023298
25%	74.930521	74.882085	74.828286
50%	82.575941	82.249916	82.292607
75%	92.147891	91.899957	91.822852
max	147.732262	138.880940	140.253340

generate_nan_means_summaries(train_abp_nanmeans, val_abp_nanmeans, test_abp_nanmeans, group='positive')

	train	validation	test
count	4916.000000	775.000000	2461.000000
mean	77.784496	77.277184	77.526598
std	10.395182	10.386473	10.244708
min	65.093372	65.146761	65.023298
25%	70.545436	70.179040	70.281144
50%	74.884225	74.526288	74.864361
75%	81.732825	80.940235	81.425981
max	146.954317	131.989181	135.669169

generate_nan_means_summaries(train_abp_nanmeans, val_abp_nanmeans, test_abp_nanmeans, group='negative')

	train	validation	test
count	8421.000000	1412.000000	4292.000000
mean	88.446668	88.210754	88.073908
std	11.378201	11.377544	11.192447
min	65.289712	65.588582	65.216607
25%	79.911913	79.678075	79.518148
50%	87.074796	86.681360	86.878588
75%	95.722421	95.251525	95.323702
max	147.732262	138.880940	140.253340

ABP Nanmean Histograms

plot_nan_means_hist(train_abp_nanmeans)

plot_nan_means_hist(val_abp_nanmeans)

plot_nan_means_hist(test_abp_nanmeans)

ABP Nanmean Scatter Plots

plot_nan_means(train_abp_nanmeans, 'Train')

plot_nan_means(val_abp_nanmeans, 'Validation')

plot_nan_means(test_abp_nanmeans, 'Test')

# Cleanup
del train_abp_nanmeans
del val_abp_nanmeans
del test_abp_nanmeans

Classification Studies

Check if data can be easily classified using non-deep learning methods. Create a balanced sample of IOH and non-IOH events and use a simple classifier to see if the data can be easily separated. Datasets which can be easily separated by non-deep learning methods should also be easily classified by deep learning models.

MAX_CLASSIFICATION_SAMPLES = 250
MAX_SAMPLE_SIZE = 1600
classification_sample_size = MAX_SAMPLE_SIZE if len(samples) >= MAX_SAMPLE_SIZE else len(samples)

classification_samples = random.sample(samples, classification_sample_size)

positive_samples = []
negative_samples = []

for sample in classification_samples:
    (sampleAbp, sampleEcg, sampleEeg) = get_segment_data(sample[0])
    
    if sample[1] == "True":
        positive_samples.append([sample[0], True, sampleAbp, sampleEcg, sampleEeg])
    else:
        negative_samples.append([sample[0], False, sampleAbp, sampleEcg, sampleEeg])

positive_samples = pd.DataFrame(positive_samples, columns=["file_path", "segment_label", "segment_abp", "segment_ecg", "segment_eeg"])
negative_samples = pd.DataFrame(negative_samples, columns=["file_path", "segment_label", "segment_abp", "segment_ecg", "segment_eeg"])

total_to_sample_pos = MAX_CLASSIFICATION_SAMPLES if len(positive_samples) >= MAX_CLASSIFICATION_SAMPLES else len(positive_samples)
total_to_sample_neg = MAX_CLASSIFICATION_SAMPLES if len(negative_samples) >= MAX_CLASSIFICATION_SAMPLES else len(negative_samples)

# Select up to 150 random samples where segment_label is True
positive_samples = positive_samples.sample(total_to_sample_pos, random_state=RANDOM_SEED)
# Select up to 150 random samples where segment_label is False
negative_samples = negative_samples.sample(total_to_sample_neg, random_state=RANDOM_SEED)

print(f'positive_samples: {len(positive_samples)}')
print(f'negative_samples: {len(negative_samples)}')

# Combine the positive and negative samples
samples_balanced = pd.concat([positive_samples, negative_samples])

positive_samples: 250
negative_samples: 250

Define function to build data for study. Each waveform field can be enabled or disabled:

def get_x_y(samples, use_abp, use_ecg, use_eeg):
    # Create X and y, using data from `samples_balanced` and the `use_abp`, `use_ecg`, and `use_eeg` variables
    X = []
    y = []
    for i in range(len(samples)):
        row = samples.iloc[i]
        sample = np.array([])
        if use_abp:
            if len(row['segment_abp']) != 30000:
                print(len(row['segment_abp']))
            sample = np.append(sample, row['segment_abp'])
        if use_ecg:
            if len(row['segment_ecg']) != 30000:
                print(len(row['segment_ecg']))
            sample = np.append(sample, row['segment_ecg'])
        if use_eeg:
            if len(row['segment_eeg']) != 7680:
                print(len(row['segment_eeg']))
            sample = np.append(sample, row['segment_eeg'])
        X.append(sample)
        # Convert the label from boolean to 0 or 1
        y.append(int(row['segment_label']))
    return X, y

KNN

Define KNN run. This is configurable to enable or disable different data channels so that we can study them individually or together:

N_NEIGHBORS = 20

def run_knn(samples, use_abp, use_ecg, use_eeg):
    # Get samples
    X,y = get_x_y(samples, use_abp, use_ecg, use_eeg)

    # Split samples into train and val
    knn_X_train, knn_X_test, knn_y_train, knn_y_test = train_test_split(X, y, test_size=0.2, random_state=RANDOM_SEED)

    # Normalize the data
    scaler = StandardScaler()
    scaler.fit(knn_X_train)

    knn_X_train = scaler.transform(knn_X_train)
    knn_X_test = scaler.transform(knn_X_test)

    # Initialize the KNN classifier
    knn = KNeighborsClassifier(n_neighbors=N_NEIGHBORS)

    # Train the KNN classifier
    knn.fit(knn_X_train, knn_y_train)

    # Make predictions on the test set
    knn_y_pred = knn.predict(knn_X_test)

    # Evaluate the KNN classifier
    print(f"ABP: {use_abp}, ECG: {use_ecg}, EEG: {use_eeg}")
    print(f"Confusion matrix:\n{confusion_matrix(knn_y_test, knn_y_pred)}")
    print(f"Classification report:\n{classification_report(knn_y_test, knn_y_pred)}")

Study each waveform independently, then ABP+EEG (which had best results in paper), and ABP+ECG+EEG:

run_knn(samples_balanced, use_abp=True, use_ecg=False, use_eeg=False)
run_knn(samples_balanced, use_abp=False, use_ecg=True, use_eeg=False)
run_knn(samples_balanced, use_abp=False, use_ecg=False, use_eeg=True)
run_knn(samples_balanced, use_abp=True, use_ecg=False, use_eeg=True)
run_knn(samples_balanced, use_abp=True, use_ecg=True, use_eeg=True)

ABP: True, ECG: False, EEG: False
Confusion matrix:
[[38 16]
 [ 9 37]]
Classification report:
              precision    recall  f1-score   support

           0       0.81      0.70      0.75        54
           1       0.70      0.80      0.75        46

    accuracy                           0.75       100
   macro avg       0.75      0.75      0.75       100
weighted avg       0.76      0.75      0.75       100

ABP: False, ECG: True, EEG: False
Confusion matrix:
[[53  1]
 [46  0]]
Classification report:
              precision    recall  f1-score   support

           0       0.54      0.98      0.69        54
           1       0.00      0.00      0.00        46

    accuracy                           0.53       100
   macro avg       0.27      0.49      0.35       100
weighted avg       0.29      0.53      0.37       100

ABP: False, ECG: False, EEG: True
Confusion matrix:
[[ 0 54]
 [ 2 44]]
Classification report:
              precision    recall  f1-score   support

           0       0.00      0.00      0.00        54
           1       0.45      0.96      0.61        46

    accuracy                           0.44       100
   macro avg       0.22      0.48      0.31       100
weighted avg       0.21      0.44      0.28       100

ABP: True, ECG: False, EEG: True
Confusion matrix:
[[39 15]
 [11 35]]
Classification report:
              precision    recall  f1-score   support

           0       0.78      0.72      0.75        54
           1       0.70      0.76      0.73        46

    accuracy                           0.74       100
   macro avg       0.74      0.74      0.74       100
weighted avg       0.74      0.74      0.74       100

ABP: True, ECG: True, EEG: True
Confusion matrix:
[[39 15]
 [11 35]]
Classification report:
              precision    recall  f1-score   support

           0       0.78      0.72      0.75        54
           1       0.70      0.76      0.73        46

    accuracy                           0.74       100
   macro avg       0.74      0.74      0.74       100
weighted avg       0.74      0.74      0.74       100

Based on the data above, the ABP data alone is strongly predictive based on the macro average F1-score of 0.90. The ECG and EEG data are weakly predictive with F1 scores of 0.33 and 0.64, respectively. The ABP+EEG data is also strongly predictive with an F1 score of 0.88, and ABP+ECG+EEG data somewhat predictive with an F1 score of 0.79.

Models based on ABP data alone, or ABP+EEG data are expected to train easily with good performance. The other signals appear to mostly add noise and are not strongly predictive. This agrees with the results from the paper.

t-SNE

Define t-SNE run. This is configurable to enable or disable different data channels so that we can study them individually or together:

def run_tsne(samples, use_abp, use_ecg, use_eeg):
    # Get samples
    X,y = get_x_y(samples, use_abp, use_ecg, use_eeg)
    
    # Convert X and y to numpy arrays
    X = np.array(X)
    y = np.array(y)

    # Run t-SNE on the samples
    tsne = TSNE(n_components=len(np.unique(y)), random_state=RANDOM_SEED)
    X_tsne = tsne.fit_transform(X)
    
    # Create a scatter plot of the t-SNE representation
    plt.figure(figsize=(16, 9))
    plt.title(f"use_abp={use_abp}, use_ecg={use_ecg}, use_eeg={use_eeg}")
    for i, label in enumerate(set(y)):
        plt.scatter(X_tsne[y == label, 0], X_tsne[y == label, 1], label=label)
    plt.legend()
    plt.show()

Study each waveform independently, then ABP+EEG (which had best results in paper), and ABP+ECG+EEG:

run_tsne(samples_balanced, use_abp=True, use_ecg=False, use_eeg=False)
run_tsne(samples_balanced, use_abp=False, use_ecg=True, use_eeg=False)
run_tsne(samples_balanced, use_abp=False, use_ecg=False, use_eeg=True)
run_tsne(samples_balanced, use_abp=True, use_ecg=False, use_eeg=True)
run_tsne(samples_balanced, use_abp=True, use_ecg=True, use_eeg=True)

Based on the plots above, it appears that ABP alone, ABP+EEG and ABP+ECG+EEG are somewhat separable, though with outliers, and should be trainable by our model. The ECG and EEG data are not easily separable from the other data. This agrees with the results from the paper.

# cleanup
del samples_balanced

Model

The model implementation is based on the CNN architecture described in Jo Y-Y et al. (2022). It is designed to handle 1, 2, or 3 signal categories simultaneously, allowing for flexible model configurations based on different combinations of physiological signals:

• ABP alone
• EEG alone
• ECG alone
• ABP + EEG
• ABP + ECG
• EEG + ECG
• ABP + EEG + ECG

Model Architecture

The architecture, as depicted in Figure 2 from the original paper, utilizes a ResNet-based approach tailored for time-series data from different physiological signals. The model architecture is adapted to handle varying input signal frequencies, with specific hyperparameters for each signal type, particularly EEG, due to its distinct characteristics compared to ABP and ECG. A diagram of the model architecture is shown below:

Each input signal is processed through a sequence of 12 7-layer residual blocks, followed by a flattening process and a linear transformation to produce a 32-dimensional feature vector per signal type. These vectors are then concatenated (if multiple signals are used) and passed through two additional linear layers to produce a single output vector, representing the IOH index. A threshold is determined experimentally in order to minimize the differene between the sensitivity and specificity and is applied to this index to perform binary classification for predicting IOH events.

The hyperparameters for the residual blocks are specified in Supplemental Table 1 from the original paper and vary for different signal type.

A forward pass through the model passes through 85 layers before concatenation, followed by two more linear layers and finally a sigmoid activation layer to produce the prediction measure.

Residual Block Definition

Each residual block consists of the following seven layers:

• Batch normalization
• ReLU
• Dropout (0.5)
• 1D convolution
• Batch normalization
• ReLU
• 1D convolution

Skip connections are included to aid in gradient flow during training, with optional 1D convolution in the skip connection to align dimensions.

Residual Block Hyperparameters

The hyperparameters are detailed in Supplemental Table 1 of the original paper. A screenshot of these hyperparameters is provided for reference below:

Note: Please be aware of a transcription error in the original paper's Supplemental Table 1 for the ECG+ABP configuration in Residual Blocks 11 and 12, where the output size should be 469 6 instead of the reported 496 6.

Training Objectives

Our model uses binary cross entropy as the loss function and Adam as the optimizer, consistent with the original study. The learning rate is set at 0.0001, and training is configured to run for up to 100 epochs, with early stopping implemented if no improvement in loss is observed over five consecutive epochs.

# First define the residual block which is reused 12x for each data track for each sample.
# Second define the primary model.
class ResidualBlock(nn.Module):
    def __init__(self, in_features: int, out_features: int, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, size_down: bool = False, ignoreSkipConnection: bool = False) -> None:
        super(ResidualBlock, self).__init__()
        
        self.ignoreSkipConnection = ignoreSkipConnection

        # calculate the appropriate padding required to ensure expected sequence lengths out of each residual block
        padding = int((((stride-1)*in_features)-stride+kernel_size)/2)

        self.size_down = size_down
        self.bn1 = nn.BatchNorm1d(in_channels)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.5)
        self.conv1 = nn.Conv1d(in_channels, out_channels, kernel_size=kernel_size, stride=1, padding=padding, bias=False)
        self.bn2 = nn.BatchNorm1d(out_channels)
        self.conv2 = nn.Conv1d(out_channels, out_channels, kernel_size=kernel_size, stride=1, padding=padding, bias=False)
        
        self.residualConv = nn.Conv1d(in_channels, out_channels, kernel_size=kernel_size, stride=1, padding=padding, bias=False)

        # unclear where in sequence this should take place. Size down expressed in Supplemental table S1
        if self.size_down:
            pool_padding = (1 if (in_features % 2 > 0) else 0)
            self.downsample = nn.MaxPool1d(kernel_size=2, stride=2, padding = pool_padding)
        
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        identity = x
        
        out = self.bn1(x)
        out = self.relu(out)
        out = self.dropout(out)
        out = self.conv1(out)

        if self.size_down:
            out = self.downsample(out)

        out = self.bn2(out)
        out = self.relu(out)
        out = self.conv2(out)
        
        if not self.ignoreSkipConnection:
          if out.shape != identity.shape:
              # run the residual through a convolution when necessary
              identity = self.residualConv(identity)
            
              outlen = np.prod(out.shape)
              idlen = np.prod(identity.shape)
              # downsample when required
              if idlen > outlen:
                  identity = self.downsample(identity)
              # match dimensions
              identity = identity.reshape(out.shape)

          # add the residual       
          out += identity

        return  out

class HypotensionCNN(nn.Module):
    def __init__(self, useAbp: bool = True, useEeg: bool = False, useEcg: bool = False, device: str = "cpu", nResiduals: int = 12, ignoreSkipConnection: bool = False, useSigmoid: bool = True) -> None:
        assert useAbp or useEeg or useEcg, "At least one data track must be used"
        assert nResiduals > 0 and nResiduals <= 12, "Number of residual blocks must be between 1 and 12"
        super(HypotensionCNN, self).__init__()

        self.device = device

        self.useAbp = useAbp
        self.useEeg = useEeg
        self.useEcg = useEcg
        self.nResiduals = nResiduals
        self.useSigmoid = useSigmoid

        # Size of the concatenated output from the residual blocks
        concatSize = 0

        if useAbp:
          self.abpBlocks = []
          self.abpMultipliers = [1, 2, 2, 2, 2, 2, 4, 4, 4, 4, 4, 6, 6]
          self.abpSizes = [30000, 15000, 15000, 7500, 7500, 3750, 3750, 1875, 1875, 938, 938, 469, 469]
          for i in range(self.nResiduals):
            downsample = i % 2 == 0
            self.abpBlocks.append(ResidualBlock(self.abpSizes[i], self.abpSizes[i+1], self.abpMultipliers[i], self.abpMultipliers[i+1], 15 if i < 6 else 7, 1, downsample, ignoreSkipConnection))
          self.abpResiduals = nn.Sequential(*self.abpBlocks)
          self.abpFc = nn.Linear(self.abpMultipliers[self.nResiduals] * self.abpSizes[self.nResiduals], 32)
          concatSize += 32
        
        if useEcg:
          self.ecgBlocks = []
          self.ecgMultipliers = [1, 2, 2, 2, 2, 2, 4, 4, 4, 4, 4, 6, 6]
          self.ecgSizes = [30000, 15000, 15000, 7500, 7500, 3750, 3750, 1875, 1875, 938, 938, 469, 469]

          for i in range(self.nResiduals):
            downsample = i % 2 == 0
            self.ecgBlocks.append(ResidualBlock(self.ecgSizes[i], self.ecgSizes[i+1], self.ecgMultipliers[i], self.ecgMultipliers[i+1], 15 if i < 6 else 7, 1, downsample, ignoreSkipConnection))
          self.ecgResiduals = nn.Sequential(*self.ecgBlocks)
          self.ecgFc = nn.Linear(self.ecgMultipliers[self.nResiduals] * self.ecgSizes[self.nResiduals], 32)
          concatSize += 32

        if useEeg:
          self.eegBlocks = []
          self.eegMultipliers = [1, 2, 2, 2, 2, 2, 4, 4, 4, 4, 4, 6, 6]
          self.eegSizes = [7680, 3840, 3840, 1920, 1920, 960, 960, 480, 480, 240, 240, 120, 120]

          for i in range(self.nResiduals):
            downsample = i % 2 == 0
            self.eegBlocks.append(ResidualBlock(self.eegSizes[i], self.eegSizes[i+1], self.eegMultipliers[i], self.eegMultipliers[i+1], 7 if i < 6 else 3, 1, downsample, ignoreSkipConnection))
          self.eegResiduals = nn.Sequential(*self.eegBlocks)
          self.eegFc = nn.Linear(self.eegMultipliers[self.nResiduals] * self.eegSizes[self.nResiduals], 32)
          concatSize += 32

        self.fullLinear1 = nn.Linear(concatSize, 16)
        self.fullLinear2 = nn.Linear(16, 1)
        self.sigmoid = nn.Sigmoid()


    def forward(self, abp: torch.Tensor, eeg: torch.Tensor, ecg: torch.Tensor) -> torch.Tensor:
        batchSize = len(abp)

        # conditionally operate ABP, EEG, and ECG networks
        tensors = []
        if self.useAbp:
          self.abpResiduals.to(self.device)
          abp = self.abpResiduals(abp)
          totalLen = np.prod(abp.shape)
          abp = torch.reshape(abp, (batchSize, int(totalLen / batchSize)))
          abp = self.abpFc(abp)
          tensors.append(abp)

        if self.useEeg:
          self.eegResiduals.to(self.device)
          eeg = self.eegResiduals(eeg)
          totalLen = np.prod(eeg.shape)
          eeg = torch.reshape(eeg, (batchSize, int(totalLen / batchSize)))
          eeg = self.eegFc(eeg)
          tensors.append(eeg)
        
        if self.useEcg:
          self.ecgResiduals.to(self.device)
          ecg = self.ecgResiduals(ecg)
          totalLen = np.prod(ecg.shape)
          ecg = torch.reshape(ecg, (batchSize, int(totalLen / batchSize)))
          ecg = self.ecgFc(ecg)
          tensors.append(ecg)

        # concatenate the tensors along dimension 1 if there's more than one, otherwise use the single tensor
        merged = torch.cat(tensors, dim=1) if len(tensors) > 1 else tensors[0]

        totalLen = np.prod(merged.shape)
        merged = torch.reshape(merged, (batchSize, int(totalLen / batchSize)))
        out = self.fullLinear1(merged)
        out = self.fullLinear2(out)
        if self.useSigmoid:
            out = self.sigmoid(out)

        # We should not be seeing NaNs! If we are, there is a problem upstream.
        #out = torch.nan_to_num(out)
        return out

Training

As discussed earlier, our model uses binary cross entropy as the loss function and Adam as the optimizer, consistent with the original study. The learning rate is set at 0.0001, and training is configured to run for up to 100 epochs, with early stopping implemented if no improvement in loss is observed over five consecutive epochs.

def train_model_one_iter(model, device, loss_func, optimizer, train_loader):
    model.train()
    train_losses = []
    
    for abp, ecg, eeg, label in tqdm(train_loader):
        batch = len(abp)
        abp = abp.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
        ecg = ecg.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
        eeg = eeg.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
        label = label.type(torch.float).reshape(batch, 1).to(device)

        optimizer.zero_grad()
        mdl = model(abp, eeg, ecg)
        loss = loss_func(torch.nan_to_num(mdl), label)
        loss.backward()
        optimizer.step()
        train_losses.append(loss.cpu().data.numpy())
    return np.mean(train_losses)

def evaluate_model(model, loss_func, val_loader):
    model.eval()
    val_losses = []
    for abp, ecg, eeg, label in tqdm(val_loader):
        batch = len(abp)

        abp = abp.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
        ecg = ecg.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
        eeg = eeg.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
        label = label.type(torch.float).reshape(batch, 1).to(device)

        mdl = model(abp, eeg, ecg)
        loss = loss_func(torch.nan_to_num(mdl), label)
        val_losses.append(loss.cpu().data.numpy())
    return np.mean(val_losses)

def plot_losses(train_losses, val_losses, best_epoch, experimentName):
    print()
    print(f'Plot Validation and Loss Values from Training')
    print(f'  Epoch with best Validation Loss:  {best_epoch:3}, {val_losses[best_epoch]:.4}')

    # Create x-axis values for epochs
    epochs = range(0, len(train_losses))

    plt.figure(figsize=(16, 9))

    # Plot the training and validation losses
    plt.plot(epochs, train_losses, 'b', label='Training Loss')
    plt.plot(epochs, val_losses, 'r', label='Validation Loss')

    # Add a vertical bar at the best_epoch
    plt.axvline(x=best_epoch, color='g', linestyle='--', label='Best Epoch')

    # Shade everything to the right of the best_epoch a light red
    plt.axvspan(best_epoch, max(epochs), facecolor='r', alpha=0.1)

    # Add labels and title
    plt.xlabel('Epochs')
    plt.ylabel('Loss')
    plt.title(experimentName)

    # Add legend
    plt.legend(loc='upper right')

    # Show the plot
    plt.show()

def eval_model(model, device, dataloader, loss_func, print_detailed: bool = False):
    model.eval()
    model = model.to(device)
    total_loss = 0
    all_predictions = []
    all_labels = []

    with torch.no_grad():
        for abp, ecg, eeg, label in tqdm(dataloader):
            batch = len(abp)
    
            abp = torch.nan_to_num(abp.reshape(batch, 1, -1)).type(torch.FloatTensor).to(device)
            ecg = torch.nan_to_num(ecg.reshape(batch, 1, -1)).type(torch.FloatTensor).to(device)
            eeg = torch.nan_to_num(eeg.reshape(batch, 1, -1)).type(torch.FloatTensor).to(device)
            label = label.type(torch.float).reshape(batch, 1).to(device)
   
            pred = model(abp, eeg, ecg)
            loss = loss_func(pred, label)
            total_loss += loss.item()

            all_predictions.append(pred.detach().cpu().numpy())
            all_labels.append(label.detach().cpu().numpy())

    # Flatten the lists
    all_predictions = np.concatenate(all_predictions).flatten()
    all_labels = np.concatenate(all_labels).flatten()

    # Calculate AUROC and AUPRC
    # y_true, y_pred
    auroc = roc_auc_score(all_labels, all_predictions)
    precision, recall, _ = precision_recall_curve(all_labels, all_predictions)
    auprc = auc(recall, precision)

    # Determine the optimal threshold, which is argmin(abs(sensitivity - specificity)) per the paper
    thresholds = np.linspace(0, 1, 101) # 0 to 1 in 0.01 steps
    min_diff = float('inf')
    optimal_sensitivity = None
    optimal_specificity = None
    optimal_threshold = None

    for threshold in thresholds:
        all_predictions_binary = (all_predictions > threshold).astype(int)

        tn, fp, fn, tp = confusion_matrix(all_labels, all_predictions_binary).ravel()
        sensitivity = tp / (tp + fn)
        specificity = tn / (tn + fp)
        diff = abs(sensitivity - specificity)

        if diff < min_diff:
            min_diff = diff
            optimal_threshold = threshold
            optimal_sensitivity = sensitivity
            optimal_specificity = specificity

    avg_loss = total_loss / len(dataloader)

    if print_detailed:
        print(f"Predictions: {all_predictions}")
        print(f"Labels: {all_labels}")
    print(f"Loss: {avg_loss}")
    print(f"AUROC: {auroc}")
    print(f"AUPRC: {auprc}")
    print(f"Sensitivity: {optimal_sensitivity}")
    print(f"Specificity: {optimal_specificity}")
    print(f"Threshold: {optimal_threshold}")

    return all_predictions, all_labels, avg_loss, auroc, auprc, optimal_sensitivity, optimal_specificity, optimal_threshold

def print_all_evals(model, models, device, val_loader, test_loader, loss_func, print_detailed: bool = False):
    print()
    print(f'Generate AUROC/AUPRC for Each Intermediate Model')
    print()
    val_aurocs = []
    val_auprcs = []

    test_aurocs = []
    test_auprcs = []

    for mod in models:
        model.load_state_dict(torch.load(mod))
        model.train(False)
        print(f'Intermediate Model:')
        print(f'  {mod}')
    
        # validation loop
        print("AUROC/AUPRC on Validation Data")
        _, _, _, valid_auroc, valid_auprc, _, _, _ = \
            eval_model(model, device, val_loader, loss_func, print_detailed)
        
        val_aurocs.append(valid_auroc)
        val_auprcs.append(valid_auprc)
        print()
    
        # test loop
        print("AUROC/AUPRC on Test Data")
        _, _, _, test_auroc, test_auprc, _, _, _ = \
            eval_model(model, device, test_loader, loss_func, print_detailed)
        
        test_aurocs.append(test_auroc)
        test_auprcs.append(test_auprc)
        print()
    
    return val_aurocs, val_auprcs, test_aurocs, test_auprcs

def plot_auroc_auprc(val_losses, val_aurocs, val_auprcs, test_aurocs, test_auprcs, all_models, best_epoch):
    print()
    print(f'Plot AUROC/AUPRC for Each Intermediate Model')
    
    # Create x-axis values for epochs
    epochs = range(0, len(val_aurocs))

    # Find model with highest AUROC
    np_test_aurocs = np.array(test_aurocs)
    test_auroc_idx = np.argmax(np_test_aurocs)

    print(f'  Epoch with best Validation Loss:  {best_epoch:3}, {val_losses[best_epoch]:.4}')
    print(f'  Epoch with best model Test AUROC: {test_auroc_idx:3}, {np.max(np_test_aurocs):.4}')
    #print(f'Best Model on Validation Loss:')
    #print(f'  {all_models[test_auroc_idx]}')
    #print(f'Best Model on Test AUROC:')
    #print(f'  {all_models[best_epoch]}')

    plt.figure(figsize=(16, 9))

    # Plot the training and validation losses
    plt.plot(epochs, val_aurocs, 'C0', label='AUROC - Validation')
    plt.plot(epochs, test_aurocs, 'C1', label='AUROC - Test')

    plt.plot(epochs, val_auprcs, 'C2', label='AUPRC - Validation')
    plt.plot(epochs, test_auprcs, 'C3', label='AUPRC - Test')

    # Add a vertical bar at the best_epoch
    plt.axvline(x=best_epoch, color='g', linestyle='--', label='Best Epoch - Validation Loss')
    plt.axvline(x=test_auroc_idx, color='maroon', linestyle='--', label='Best Epoch - Test AUROC')

    # Shade everything to the right of the best_model a light red
    plt.axvspan(test_auroc_idx, max(epochs), facecolor='r', alpha=0.1)

    # Add labels and title
    plt.xlabel('Epochs')
    plt.ylabel('AUROC / AUPRC')
    plt.title('Validation and Test AUROC by Model Iteration Across Training')

    # Add legend
    plt.legend(loc='right')

    # Show the plot
    plt.show()

    return np_test_aurocs, test_auroc_idx

def run_experiment(
    experimentNamePrefix: str = None,
    useAbp: bool = True, 
    useEeg: bool = False, 
    useEcg: bool = False, 
    nResiduals: int = 12, 
    skip_connection: bool = False, 
    batch_size: int = 64, 
    learning_rate: float = 1e-4, 
    weight_decay: float = 0.0, 
    balance_labels: bool = False,
    pos_weight: float = None,
    max_epochs: int = 100, 
    patience: int = 25, 
    device: str = "cpu"
):
    experimentName = ""

    experimentOptions = [experimentNamePrefix, 'ABP', 'EEG', 'ECG', 'SKIPCONNECTION']
    experimentValues = [experimentNamePrefix is not None, useAbp, useEeg, useEcg, skip_connection]
    experimentFlags = [name for name, value in zip(experimentOptions, experimentValues) if value]
    if experimentFlags:
        experimentName = "_".join(experimentFlags)

    experimentName = f"{experimentName}_{nResiduals}_RESIDUAL_BLOCKS_{batch_size}_BATCH_SIZE_{learning_rate}_LEARNING_RATE"

    if weight_decay is not None and weight_decay != 0.0:
        experimentName = f"{experimentName}_{weight_decay}_WEIGHT_DECAY"

    predictionWindow = 'ALL' if PREDICTION_WINDOW == 'ALL' else f'{PREDICTION_WINDOW:03}'
    experimentName = f"{experimentName}_{predictionWindow}_MINS"

    maxCases = '_ALL' if MAX_CASES is None else f'{MAX_CASES:04}'
    experimentName = f"{experimentName}_{maxCases}_MAX_CASES"
    
    # default label split based on empirical data
    my_pos_weight = 4.0
    if balance_labels and pos_weight is not None:
        my_pos_weight = pos_weight

    print(f"Experiment Setup")
    print(f'  name:              {experimentName}')
    print(f'  prediction_window: {predictionWindow}')
    print(f'  max_cases:         {maxCases}')
    print(f'  use_abp:           {useAbp}')
    print(f'  use_eeg:           {useEeg}')
    print(f'  use_ecg:           {useEcg}')
    print(f'  n_residuals:       {nResiduals}')
    print(f'  skip_connection:   {skip_connection}')
    print(f'  batch_size:        {batch_size}')
    print(f'  learning_rate:     {learning_rate}')
    print(f'  weight_decay:      {weight_decay}')
    print(f'  balance_labels:    {balance_labels}')
    if balance_labels:
        print(f'  pos_weight:        {my_pos_weight}')
    print(f'  max_epochs:        {max_epochs}')
    print(f'  patience:          {patience}')
    print(f'  device:            {device}')
    print()

    train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
    val_loader = torch.utils.data.DataLoader(val_dataset, batch_size=batch_size, shuffle=True)
    test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=False)

    # Disable final sigmoid activation for BCEWithLogitsLoss
    model = HypotensionCNN(useAbp, useEeg, useEcg, device, nResiduals, skip_connection, useSigmoid=(not balance_labels))
    model = model.to(device)
    
    if balance_labels:
        # Only the weight for the positive class
        loss_func = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([my_pos_weight]).to(device))
    else:
        loss_func = nn.BCELoss()
    optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate, weight_decay=weight_decay)

    
    print(f'Model Architecture')
    print(model)
    print()

    print(f'Training Loop')
    # Training loop
    best_epoch = 0
    train_losses = []
    val_losses = []
    best_loss = float('inf')
    no_improve_epochs = 0
    model_path = os.path.join(VITAL_MODELS, f"{experimentName}.model")

    all_models = []

    for i in range(max_epochs):
        # Train the model and get the training loss
        train_loss = train_model_one_iter(model, device, loss_func, optimizer, train_loader)
        train_losses.append(train_loss)
        # Calculate validate loss
        val_loss = evaluate_model(model, loss_func, val_loader)
        val_losses.append(val_loss)
        print(f"[{datetime.now()}] Completed epoch {i} with training loss {train_loss:.8f}, validation loss {val_loss:.8f}")

        # Save all intermediary models.
        tmp_model_path = os.path.join(VITAL_MODELS, f"{experimentName}_{i:04d}.model")
        torch.save(model.state_dict(), tmp_model_path)
        all_models.append(tmp_model_path)
  
        # Check if validation loss has improved
        if val_loss < best_loss:
            best_epoch = i
            best_loss = val_loss
            no_improve_epochs = 0
            torch.save(model.state_dict(), model_path)
            print(f"Validation loss improved to {val_loss:.8f}. Model saved.")
        else:
            no_improve_epochs += 1
            print(f"No improvement in validation loss. {no_improve_epochs} epochs without improvement.")

        # exit early if no improvement in loss over last 'patience' epochs
        if no_improve_epochs >= patience:
            print("Early stopping due to no improvement in validation loss.")
            break

    # Load best model from disk
    #print()
    #if os.path.exists(model_path):
    #    model.load_state_dict(torch.load(model_path))
    #    print(f"Loaded best model from disk from epoch {best_epoch}.")
    #else:
    #    print("No saved model found for f{experimentName}.")

    model.train(False)

    # Plot the training and validation losses across all training epochs.
    plot_losses(train_losses, val_losses, best_epoch, experimentName)

    # Generate AUROC/AUPRC for each intermediate model generated across training epochs.
    val_aurocs, val_auprcs, test_aurocs, test_auprcs = \
        print_all_evals(model, all_models, device, val_loader, test_loader, loss_func, print_detailed=False)

    # Find model with highest AUROC. Plot AUROC/AUPRC across all epochs.
    np_test_aurocs, test_auroc_idx = plot_auroc_auprc(val_losses, val_aurocs, val_auprcs, \
                                      test_aurocs, test_auprcs, all_models, best_epoch)

    ## AUROC / AUPRC - Model with Best Validation Loss
    best_model_val_loss = all_models[best_epoch]
    
    print(f'AUROC/AUPRC Plots - Best Model Based on Validation Loss')
    print(f'  Epoch with best Validation Loss:  {best_epoch:3}, {val_losses[best_epoch]:.4}')
    print(f'  Best Model Based on Validation Loss:')
    print(f'    {best_model_val_loss}')
    print()
    print(f'Generate Stats Based on Test Data')
    model.load_state_dict(torch.load(best_model_val_loss))
    model.train(False)
    
    best_model_val_test_predictions, best_model_val_test_labels, test_loss, \
        best_model_val_test_auroc, best_model_val_test_auprc, test_sensitivity, test_specificity, \
        best_model_val_test_threshold = eval_model(model, device, test_loader, loss_func, print_detailed=False)

    # y_test, y_pred
    display = RocCurveDisplay.from_predictions(
        best_model_val_test_labels,
        best_model_val_test_predictions,
        plot_chance_level=True
    )
    plt.show()

    print(f'best_model_val_test_auroc: {best_model_val_test_auroc}')

    best_model_val_test_predictions_binary = \
    (best_model_val_test_predictions > best_model_val_test_threshold).astype(int)

    # y_test, y_pred
    display = PrecisionRecallDisplay.from_predictions(
        best_model_val_test_labels, 
        best_model_val_test_predictions_binary,
        plot_chance_level=True
    )
    plt.show()

    print(f'best_model_val_test_auprc: {best_model_val_test_auprc}')
    print()

    ## AUROC / AUPRC - Model with Best AUROC
    # Find model with highest AUROC
    best_model_auroc = all_models[test_auroc_idx]

    print(f'AUROC/AUPRC Plots - Best Model Based on Model AUROC')
    print(f'  Epoch with best model Test AUROC: {test_auroc_idx:3}, {np.max(np_test_aurocs):.4}')
    print(f'  Best Model Based on Model AUROC:')
    print(f'    {best_model_auroc}')
    print()
    print(f'Generate Stats Based on Test Data')
    model.load_state_dict(torch.load(best_model_auroc))
    model.train(False)
    
    best_model_auroc_test_predictions, best_model_auroc_test_labels, test_loss, \
        best_model_auroc_test_auroc, best_model_auroc_test_auprc, test_sensitivity, test_specificity, \
        best_model_auroc_test_threshold = eval_model(model, device, test_loader, loss_func, print_detailed=False)

    # y_test, y_pred
    display = RocCurveDisplay.from_predictions(
        best_model_auroc_test_labels,
        best_model_auroc_test_predictions,
        plot_chance_level=True
    )
    plt.show()

    print(f'best_model_auroc_test_auroc: {best_model_auroc_test_auroc}')

    best_model_auroc_test_predictions_binary = \
        (best_model_auroc_test_predictions > best_model_auroc_test_threshold).astype(int)

    # y_test, y_pred
    display = PrecisionRecallDisplay.from_predictions(
        best_model_auroc_test_labels, 
        best_model_auroc_test_predictions_binary,
        plot_chance_level=True
    )
    plt.show()

    print(f"best_model_auroc_test_auprc: {best_model_auroc_test_auprc}")

print('Time to experiment!')

Time to experiment!

run_experiment(
    experimentNamePrefix=None, 
    useAbp=True, 
    useEeg=False, 
    useEcg=False,
    nResiduals=12, 
    skip_connection=False,
    batch_size=64,
    learning_rate=1e-4,
    weight_decay=0.0,
    balance_labels=False,
    #pos_weight=2.0,
    pos_weight=None,
    max_epochs=100,
    patience=15,
    device=device
)

Experiment Setup
  name:              ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES
  prediction_window: 003
  max_cases:         _ALL
  use_abp:           True
  use_eeg:           False
  use_ecg:           False
  n_residuals:       12
  skip_connection:   False
  batch_size:        64
  learning_rate:     0.0001
  weight_decay:      0.0
  balance_labels:    False
  max_epochs:        100
  patience:          15
  device:            mps

Model Architecture
HypotensionCNN(
  (abpResiduals): Sequential(
    (0): ResidualBlock(
      (bn1): BatchNorm1d(1, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(1, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (residualConv): Conv1d(1, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (1): ResidualBlock(
      (bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (residualConv): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
    )
    (2): ResidualBlock(
      (bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (residualConv): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (3): ResidualBlock(
      (bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (residualConv): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
    )
    (4): ResidualBlock(
      (bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (residualConv): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (5): ResidualBlock(
      (bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(2, 4, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(4, 4, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
      (residualConv): Conv1d(2, 4, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
    )
    (6): ResidualBlock(
      (bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (residualConv): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (7): ResidualBlock(
      (bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (residualConv): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
    )
    (8): ResidualBlock(
      (bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (residualConv): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (downsample): MaxPool1d(kernel_size=2, stride=2, padding=1, dilation=1, ceil_mode=False)
    )
    (9): ResidualBlock(
      (bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (residualConv): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
    )
    (10): ResidualBlock(
      (bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(4, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (bn2): BatchNorm1d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(6, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (residualConv): Conv1d(4, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (11): ResidualBlock(
      (bn1): BatchNorm1d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU()
      (dropout): Dropout(p=0.5, inplace=False)
      (conv1): Conv1d(6, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (bn2): BatchNorm1d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (conv2): Conv1d(6, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
      (residualConv): Conv1d(6, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
    )
  )
  (abpFc): Linear(in_features=2814, out_features=32, bias=True)
  (fullLinear1): Linear(in_features=32, out_features=16, bias=True)
  (fullLinear2): Linear(in_features=16, out_features=1, bias=True)
  (sigmoid): Sigmoid()
)

Training Loop

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  3.96it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00,  4.95it/s]

[2024-05-01 05:52:15.939974] Completed epoch 0 with training loss 0.54845905, validation loss 0.59052509
Validation loss improved to 0.59052509. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 05:53:14.869796] Completed epoch 1 with training loss 0.51682955, validation loss 0.65855759
No improvement in validation loss. 1 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 05:54:13.965398] Completed epoch 2 with training loss 0.51347697, validation loss 0.62033492
No improvement in validation loss. 2 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

[2024-05-01 05:55:12.682862] Completed epoch 3 with training loss 0.50908750, validation loss 0.55703592
Validation loss improved to 0.55703592. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  3.99it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 05:56:11.883987] Completed epoch 4 with training loss 0.50609565, validation loss 0.55105549
Validation loss improved to 0.55105549. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 05:57:10.977086] Completed epoch 5 with training loss 0.50368357, validation loss 0.62956876
No improvement in validation loss. 1 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 05:58:09.789318] Completed epoch 6 with training loss 0.50122613, validation loss 0.58714724
No improvement in validation loss. 2 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 05:59:08.739627] Completed epoch 7 with training loss 0.50275570, validation loss 0.55850273
No improvement in validation loss. 3 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:00:07.827975] Completed epoch 8 with training loss 0.49952757, validation loss 0.57284427
No improvement in validation loss. 4 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

[2024-05-01 06:01:06.902822] Completed epoch 9 with training loss 0.49783376, validation loss 0.59470636
No improvement in validation loss. 5 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

[2024-05-01 06:02:05.837565] Completed epoch 10 with training loss 0.49747616, validation loss 0.57383054
No improvement in validation loss. 6 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:03:04.819358] Completed epoch 11 with training loss 0.49439850, validation loss 0.55042309
Validation loss improved to 0.55042309. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.16it/s]

[2024-05-01 06:04:03.688742] Completed epoch 12 with training loss 0.49488166, validation loss 0.57511932
No improvement in validation loss. 1 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:05:02.369610] Completed epoch 13 with training loss 0.49084646, validation loss 0.54866123
Validation loss improved to 0.54866123. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.16it/s]

[2024-05-01 06:06:01.379038] Completed epoch 14 with training loss 0.49175668, validation loss 0.55878216
No improvement in validation loss. 1 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:07:00.245122] Completed epoch 15 with training loss 0.48997128, validation loss 0.55655724
No improvement in validation loss. 2 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:07:59.254088] Completed epoch 16 with training loss 0.48890623, validation loss 0.56272739
No improvement in validation loss. 3 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.17it/s]

[2024-05-01 06:08:58.201056] Completed epoch 17 with training loss 0.48559666, validation loss 0.56365335
No improvement in validation loss. 4 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:09:56.901396] Completed epoch 18 with training loss 0.48504612, validation loss 0.56514549
No improvement in validation loss. 5 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:10:55.858487] Completed epoch 19 with training loss 0.48484984, validation loss 0.54416662
Validation loss improved to 0.54416662. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

[2024-05-01 06:11:54.952033] Completed epoch 20 with training loss 0.48363432, validation loss 0.54701400
No improvement in validation loss. 1 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:12:53.694357] Completed epoch 21 with training loss 0.48158345, validation loss 0.55995244
No improvement in validation loss. 2 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

[2024-05-01 06:13:52.836525] Completed epoch 22 with training loss 0.48322493, validation loss 0.57864243
No improvement in validation loss. 3 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:14:51.849248] Completed epoch 23 with training loss 0.48168632, validation loss 0.56460971
No improvement in validation loss. 4 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

[2024-05-01 06:15:50.600099] Completed epoch 24 with training loss 0.47952753, validation loss 0.55760300
No improvement in validation loss. 5 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:16:49.509678] Completed epoch 25 with training loss 0.48034054, validation loss 0.56507963
No improvement in validation loss. 6 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.08it/s]

[2024-05-01 06:17:48.594314] Completed epoch 26 with training loss 0.47821549, validation loss 0.54509956
No improvement in validation loss. 7 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.04it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.16it/s]

[2024-05-01 06:18:47.205087] Completed epoch 27 with training loss 0.47935650, validation loss 0.57744205
No improvement in validation loss. 8 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.04it/s]

[2024-05-01 06:19:46.323120] Completed epoch 28 with training loss 0.47750622, validation loss 0.54066294
Validation loss improved to 0.54066294. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:20:45.363493] Completed epoch 29 with training loss 0.47652009, validation loss 0.54733419
No improvement in validation loss. 1 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.08it/s]

[2024-05-01 06:21:44.104325] Completed epoch 30 with training loss 0.47453353, validation loss 0.58113801
No improvement in validation loss. 2 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.16it/s]

[2024-05-01 06:22:43.089748] Completed epoch 31 with training loss 0.47749081, validation loss 0.54751712
No improvement in validation loss. 3 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.17it/s]

[2024-05-01 06:23:42.092817] Completed epoch 32 with training loss 0.47334585, validation loss 0.55001366
No improvement in validation loss. 4 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:24:40.943000] Completed epoch 33 with training loss 0.47331816, validation loss 0.55479157
No improvement in validation loss. 5 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:25:39.879635] Completed epoch 34 with training loss 0.47258097, validation loss 0.54148060
No improvement in validation loss. 6 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.16it/s]

[2024-05-01 06:26:38.735940] Completed epoch 35 with training loss 0.47086498, validation loss 0.54867750
No improvement in validation loss. 7 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.16it/s]

[2024-05-01 06:27:37.474818] Completed epoch 36 with training loss 0.47133112, validation loss 0.59381622
No improvement in validation loss. 8 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:28:36.592718] Completed epoch 37 with training loss 0.46956050, validation loss 0.55694813
No improvement in validation loss. 9 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:29:35.410165] Completed epoch 38 with training loss 0.46869338, validation loss 0.56618899
No improvement in validation loss. 10 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  3.99it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.17it/s]

[2024-05-01 06:30:34.549959] Completed epoch 39 with training loss 0.46828377, validation loss 0.54067415
No improvement in validation loss. 11 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 06:31:33.626364] Completed epoch 40 with training loss 0.46593815, validation loss 0.53823632
Validation loss improved to 0.53823632. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:32:32.372382] Completed epoch 41 with training loss 0.46646926, validation loss 0.58251399
No improvement in validation loss. 1 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:33:31.225274] Completed epoch 42 with training loss 0.46279860, validation loss 0.53783387
Validation loss improved to 0.53783387. Model saved.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.00it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 06:34:30.365500] Completed epoch 43 with training loss 0.46181706, validation loss 0.57645744
No improvement in validation loss. 1 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 06:35:29.038430] Completed epoch 44 with training loss 0.46022037, validation loss 0.61514252
No improvement in validation loss. 2 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:36:27.686253] Completed epoch 45 with training loss 0.45952812, validation loss 0.55357546
No improvement in validation loss. 3 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.04it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:37:26.237294] Completed epoch 46 with training loss 0.45959979, validation loss 0.56573421
No improvement in validation loss. 4 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:38:24.946409] Completed epoch 47 with training loss 0.46286428, validation loss 0.56414813
No improvement in validation loss. 5 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:39:23.784364] Completed epoch 48 with training loss 0.45756015, validation loss 0.59026152
No improvement in validation loss. 6 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00,  4.01it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 06:40:22.709512] Completed epoch 49 with training loss 0.45804489, validation loss 0.54794979
No improvement in validation loss. 7 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.16it/s]

[2024-05-01 06:41:21.370362] Completed epoch 50 with training loss 0.45386106, validation loss 0.56097811
No improvement in validation loss. 8 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 06:42:20.165502] Completed epoch 51 with training loss 0.45701405, validation loss 0.60055518
No improvement in validation loss. 9 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.05it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:43:18.627322] Completed epoch 52 with training loss 0.45644122, validation loss 0.59112924
No improvement in validation loss. 10 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 06:44:17.415546] Completed epoch 53 with training loss 0.45395449, validation loss 0.61414361
No improvement in validation loss. 11 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.04it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.14it/s]

[2024-05-01 06:45:16.043160] Completed epoch 54 with training loss 0.45308992, validation loss 0.54727173
No improvement in validation loss. 12 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.02it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 06:46:14.848703] Completed epoch 55 with training loss 0.45197782, validation loss 0.57491523
No improvement in validation loss. 13 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

[2024-05-01 06:47:13.523378] Completed epoch 56 with training loss 0.45006117, validation loss 0.58290470
No improvement in validation loss. 14 epochs without improvement.

100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00,  4.03it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.15it/s]

[2024-05-01 06:48:12.263781] Completed epoch 57 with training loss 0.45042974, validation loss 0.55502719
No improvement in validation loss. 15 epochs without improvement.
Early stopping due to no improvement in validation loss.

Plot Validation and Loss Values from Training
  Epoch with best Validation Loss:   42, 0.5378

Generate AUROC/AUPRC for Each Intermediate Model

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0000.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.73it/s]

Loss: 0.5859471900122506
AUROC: 0.8035949922324773
AUPRC: 0.7038041366388864
Sensitivity: 0.7161290322580646
Specificity: 0.7407932011331445
Threshold: 0.25

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.94it/s]

Loss: 0.6083608744841702
AUROC: 0.7939504925486234
AUPRC: 0.7157216571132787
Sensitivity: 0.716781796017879
Specificity: 0.7313606710158435
Threshold: 0.24

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0001.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.06it/s]

Loss: 0.6574152963502067
AUROC: 0.8078506808005117
AUPRC: 0.7093965725413414
Sensitivity: 0.7367741935483871
Specificity: 0.7315864022662889
Threshold: 0.15

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.02it/s]

Loss: 0.6902545052316954
AUROC: 0.7967412321876445
AUPRC: 0.7205366976873793
Sensitivity: 0.7326290125965055
Specificity: 0.7134203168685928
Threshold: 0.14

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0002.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.09it/s]

Loss: 0.6226890529905047
AUROC: 0.8079119071552592
AUPRC: 0.7081702299849927
Sensitivity: 0.7303225806451613
Specificity: 0.740084985835694
Threshold: 0.18

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.88it/s]

Loss: 0.6511145466060009
AUROC: 0.7967217767726392
AUPRC: 0.7199543664234093
Sensitivity: 0.7245022348638764
Specificity: 0.7215750232991612
Threshold: 0.17

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0003.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00,  4.81it/s]

Loss: 0.5725380327020373
AUROC: 0.8070858082792651
AUPRC: 0.7058901550885577
Sensitivity: 0.7290322580645161
Specificity: 0.7415014164305949
Threshold: 0.28

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.5797299520587021
AUROC: 0.7952703365417569
AUPRC: 0.717508215000471
Sensitivity: 0.7216578626574564
Specificity: 0.7271668219944082
Threshold: 0.27

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0004.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

Loss: 0.5600509328501565
AUROC: 0.8067979530293337
AUPRC: 0.7051440989632735
Sensitivity: 0.7354838709677419
Specificity: 0.7330028328611898
Threshold: 0.29

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.06it/s]

Loss: 0.5746631549214417
AUROC: 0.7947537029666526
AUPRC: 0.7167484861064243
Sensitivity: 0.728565623730191
Specificity: 0.717381174277726
Threshold: 0.28

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0005.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.13it/s]

Loss: 0.6199249812534877
AUROC: 0.8086292607146122
AUPRC: 0.7081687466359474
Sensitivity: 0.7380645161290322
Specificity: 0.7280453257790368
Threshold: 0.17

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.6545365085860468
AUROC: 0.7964395549131218
AUPRC: 0.7179992459583092
Sensitivity: 0.7180008126777733
Specificity: 0.7374184529356943
Threshold: 0.17

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0006.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.09it/s]

Loss: 0.5778036177158355
AUROC: 0.8070519967102258
AUPRC: 0.7064476366545598
Sensitivity: 0.7406451612903225
Specificity: 0.7344192634560907
Threshold: 0.23

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.03it/s]

Loss: 0.6084165859897182
AUROC: 0.7949990968143107
AUPRC: 0.7162418904878575
Sensitivity: 0.7273466070702966
Specificity: 0.7197110904007455
Threshold: 0.22

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0007.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.01it/s]

Loss: 0.562591245344707
AUROC: 0.8059298181485881
AUPRC: 0.7066158010613205
Sensitivity: 0.7316129032258064
Specificity: 0.7443342776203966
Threshold: 0.29

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.02it/s]

Loss: 0.5772552568957491
AUROC: 0.7934264744364368
AUPRC: 0.7151381454828599
Sensitivity: 0.7139374238114587
Specificity: 0.7329916123019571
Threshold: 0.28

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0008.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.09it/s]

Loss: 0.564673832484654
AUROC: 0.805629169331993
AUPRC: 0.7070136066002808
Sensitivity: 0.743225806451613
Specificity: 0.7330028328611898
Threshold: 0.28

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.03it/s]

Loss: 0.5854992335135082
AUROC: 0.7919913180565564
AUPRC: 0.7106699701122675
Sensitivity: 0.7289719626168224
Specificity: 0.7143522833178005
Threshold: 0.27

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0009.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

Loss: 0.6133006725992475
AUROC: 0.8072000365530476
AUPRC: 0.7097292973082021
Sensitivity: 0.7406451612903225
Specificity: 0.7351274787535411
Threshold: 0.21

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.03it/s]

Loss: 0.6247424463618476
AUROC: 0.7932481568006096
AUPRC: 0.7127362663016106
Sensitivity: 0.7273466070702966
Specificity: 0.7183131407269339
Threshold: 0.2

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0010.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

Loss: 0.5694470907960619
AUROC: 0.8063684547199124
AUPRC: 0.7088218248978105
Sensitivity: 0.7406451612903225
Specificity: 0.7330028328611898
Threshold: 0.25

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.02it/s]

Loss: 0.594383901020266
AUROC: 0.7921538725459194
AUPRC: 0.7108029858030867
Sensitivity: 0.7184071515644047
Specificity: 0.7287977632805219
Threshold: 0.25

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0011.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.09it/s]

Loss: 0.5581223368644714
AUROC: 0.8070538243626063
AUPRC: 0.7062225913641743
Sensitivity: 0.7341935483870968
Specificity: 0.7407932011331445
Threshold: 0.3

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.02it/s]

Loss: 0.5705225805066666
AUROC: 0.7920145130768791
AUPRC: 0.7090080756646415
Sensitivity: 0.7228768793173507
Specificity: 0.7220410065237651
Threshold: 0.29

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0012.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

Loss: 0.5769370462213244
AUROC: 0.8073690943982454
AUPRC: 0.7085046652374707
Sensitivity: 0.7393548387096774
Specificity: 0.7386685552407932
Threshold: 0.25

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.5954774980837444
AUROC: 0.7910610557312907
AUPRC: 0.7068395544474011
Sensitivity: 0.7139374238114587
Specificity: 0.7297297297297297
Threshold: 0.25

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0013.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5498952754906246
AUROC: 0.80374577355387
AUPRC: 0.7053207258327664
Sensitivity: 0.7393548387096774
Specificity: 0.7358356940509915
Threshold: 0.31

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.06it/s]

Loss: 0.570091428902914
AUROC: 0.7883626701425746
AUPRC: 0.7050630023487928
Sensitivity: 0.712312068264933
Specificity: 0.7257688723205965
Threshold: 0.31

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0014.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

Loss: 0.5611233958176204
AUROC: 0.8049977154345243
AUPRC: 0.7047176175952845
Sensitivity: 0.7380645161290322
Specificity: 0.7372521246458924
Threshold: 0.28

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.5802073644579582
AUROC: 0.7883992614705528
AUPRC: 0.7029214581590661
Sensitivity: 0.7127184071515644
Specificity: 0.7215750232991612
Threshold: 0.28

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0015.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.08it/s]

Loss: 0.5585595897265843
AUROC: 0.8030256785159463
AUPRC: 0.7025061807781666
Sensitivity: 0.727741935483871
Specificity: 0.7450424929178471
Threshold: 0.29

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:22<00:00,  4.76it/s]

Loss: 0.5811271973938312
AUROC: 0.7860195943957801
AUPRC: 0.6982504170101814
Sensitivity: 0.7082486793986185
Specificity: 0.72856477166822
Threshold: 0.29

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0016.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00,  4.93it/s]

Loss: 0.5636099057538169
AUROC: 0.8058192451795668
AUPRC: 0.7056834488288954
Sensitivity: 0.7393548387096774
Specificity: 0.7365439093484419
Threshold: 0.27

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.96it/s]

Loss: 0.5838960510379864
AUROC: 0.7880805429566096
AUPRC: 0.7006285068504388
Sensitivity: 0.7143437626980902
Specificity: 0.7253028890959925
Threshold: 0.27

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0017.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.02it/s]

Loss: 0.5634740301540919
AUROC: 0.8058512290962259
AUPRC: 0.7054162528617351
Sensitivity: 0.7367741935483871
Specificity: 0.7422096317280453
Threshold: 0.26

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.01it/s]

Loss: 0.5902659772139676
AUROC: 0.788095548714655
AUPRC: 0.6998786104902175
Sensitivity: 0.7143437626980902
Specificity: 0.7267008387698043
Threshold: 0.26

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0018.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.06it/s]

Loss: 0.5628924182483128
AUROC: 0.8062112766151878
AUPRC: 0.7042849880875
Sensitivity: 0.7419354838709677
Specificity: 0.7337110481586402
Threshold: 0.25

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.98it/s]

Loss: 0.592963512635456
AUROC: 0.7878858467962281
AUPRC: 0.6982225982157145
Sensitivity: 0.7204388459975619
Specificity: 0.7176141658900279
Threshold: 0.25

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0019.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.00it/s]

Loss: 0.5464371119226729
AUROC: 0.7986968838526912
AUPRC: 0.68996813509794
Sensitivity: 0.7406451612903225
Specificity: 0.7273371104815864
Threshold: 0.36

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.96it/s]

Loss: 0.5600495630840086
AUROC: 0.7821669015201921
AUPRC: 0.6853861735625737
Sensitivity: 0.7151564404713531
Specificity: 0.7103914259086673
Threshold: 0.36

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0020.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.08it/s]

Loss: 0.5495232633181981
AUROC: 0.803151786530202
AUPRC: 0.6989712330591584
Sensitivity: 0.7329032258064516
Specificity: 0.7415014164305949
Threshold: 0.31

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.93it/s]

Loss: 0.567583792052179
AUROC: 0.7850250013917013
AUPRC: 0.6921835864926364
Sensitivity: 0.724095895977245
Specificity: 0.706663560111836
Threshold: 0.3

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0021.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.03it/s]

Loss: 0.5650256276130676
AUROC: 0.8047683450607694
AUPRC: 0.7002016638675831
Sensitivity: 0.7406451612903225
Specificity: 0.7337110481586402
Threshold: 0.27

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.95it/s]

Loss: 0.5806394601767918
AUROC: 0.7850549182342398
AUPRC: 0.6908906882457168
Sensitivity: 0.7184071515644047
Specificity: 0.717148182665424
Threshold: 0.27

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0022.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.02it/s]

Loss: 0.5743089182036264
AUROC: 0.8080544640409394
AUPRC: 0.7035868991249145
Sensitivity: 0.7445161290322581
Specificity: 0.7351274787535411
Threshold: 0.22

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.97it/s]

Loss: 0.6047967499438321
AUROC: 0.7879760233548292
AUPRC: 0.6957865412328825
Sensitivity: 0.7208451848841935
Specificity: 0.7190121155638397
Threshold: 0.22

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0023.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00,  4.89it/s]

Loss: 0.5672739991119929
AUROC: 0.8063474367175362
AUPRC: 0.7021889132817944
Sensitivity: 0.7458064516129033
Specificity: 0.7322946175637394
Threshold: 0.23

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.94it/s]

Loss: 0.5984816556831576
AUROC: 0.7864378621500061
AUPRC: 0.6935190437345973
Sensitivity: 0.724095895977245
Specificity: 0.7103914259086673
Threshold: 0.23

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0024.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.01it/s]

Loss: 0.553110134601593
AUROC: 0.8054496024856073
AUPRC: 0.6974905780082327
Sensitivity: 0.7367741935483871
Specificity: 0.7464589235127479
Threshold: 0.25

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.00it/s]

Loss: 0.585744377014772
AUROC: 0.7857363311271871
AUPRC: 0.6905640573334084
Sensitivity: 0.7163754571312475
Specificity: 0.7206430568499534
Threshold: 0.25

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0025.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.03it/s]

Loss: 0.5576985061168671
AUROC: 0.8118989308233574
AUPRC: 0.7029533103893714
Sensitivity: 0.7458064516129033
Specificity: 0.7464589235127479
Threshold: 0.22

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.88it/s]

Loss: 0.5950879606435884
AUROC: 0.7910940968010564
AUPRC: 0.697151972063846
Sensitivity: 0.716781796017879
Specificity: 0.7248369058713886
Threshold: 0.22

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0026.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.03it/s]

Loss: 0.5549319905894143
AUROC: 0.8081339669194919
AUPRC: 0.6974055493821754
Sensitivity: 0.7445161290322581
Specificity: 0.7386685552407932
Threshold: 0.26

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  4.98it/s]

Loss: 0.5776107519302728
AUROC: 0.7850932610229364
AUPRC: 0.68671659734642
Sensitivity: 0.7184071515644047
Specificity: 0.7166821994408201
Threshold: 0.26

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0027.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00,  5.00it/s]

Loss: 0.5719426946980612
AUROC: 0.8046248743488988
AUPRC: 0.6947635853432876
Sensitivity: 0.7445161290322581
Specificity: 0.7351274787535411
Threshold: 0.21

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:22<00:00,  4.81it/s]

Loss: 0.6098847259890359
AUROC: 0.7813553598295573
AUPRC: 0.6836268871625606
Sensitivity: 0.7216578626574564
Specificity: 0.7031686859273066
Threshold: 0.21

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0028.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00,  4.92it/s]

Loss: 0.5487099630492074
AUROC: 0.8051594626702
AUPRC: 0.6937846237361699
Sensitivity: 0.7380645161290322
Specificity: 0.7450424929178471
Threshold: 0.28

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:22<00:00,  4.79it/s]

Loss: 0.5722965238229284
AUROC: 0.7817214151196693
AUPRC: 0.6811380012342292
Sensitivity: 0.7102803738317757
Specificity: 0.717381174277726
Threshold: 0.28

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0029.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00,  4.93it/s]

Loss: 0.5505596595151084
AUROC: 0.809671022571507
AUPRC: 0.6983587729207007
Sensitivity: 0.7509677419354839
Specificity: 0.7365439093484419
Threshold: 0.24

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.06it/s]

Loss: 0.5812201449331248
AUROC: 0.7848498079831012
AUPRC: 0.6876830968719401
Sensitivity: 0.7257212515237709
Specificity: 0.7087604846225536
Threshold: 0.24

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0030.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

Loss: 0.5660754327263151
AUROC: 0.8061262907794939
AUPRC: 0.6966148572809036
Sensitivity: 0.7458064516129033
Specificity: 0.7436260623229461
Threshold: 0.21

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.6084756879311688
AUROC: 0.7828351547893646
AUPRC: 0.6891518846583943
Sensitivity: 0.7135310849248273
Specificity: 0.71784715750233
Threshold: 0.21

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0031.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.09it/s]

Loss: 0.5451005203383309
AUROC: 0.8120515397971305
AUPRC: 0.7017753683626013
Sensitivity: 0.7470967741935484
Specificity: 0.740084985835694
Threshold: 0.24

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.02it/s]

Loss: 0.5791899400218478
AUROC: 0.7882354288882333
AUPRC: 0.6927611487329255
Sensitivity: 0.7204388459975619
Specificity: 0.7213420316868593
Threshold: 0.24

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0032.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5467198865754264
AUROC: 0.811010691766426
AUPRC: 0.6948754131554438
Sensitivity: 0.7445161290322581
Specificity: 0.7450424929178471
Threshold: 0.24

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.5831167169337003
AUROC: 0.7863374135109763
AUPRC: 0.6860904570947859
Sensitivity: 0.7135310849248273
Specificity: 0.7287977632805219
Threshold: 0.24

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0033.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

Loss: 0.5557353377342225
AUROC: 0.804773828017911
AUPRC: 0.6887814305584704
Sensitivity: 0.7329032258064516
Specificity: 0.7436260623229461
Threshold: 0.24

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.5917413386533845
AUROC: 0.7779493841106726
AUPRC: 0.6751847707990541
Sensitivity: 0.7082486793986185
Specificity: 0.7176141658900279
Threshold: 0.24

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0034.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5426491243498666
AUROC: 0.7966709311888879
AUPRC: 0.6772665266566317
Sensitivity: 0.7341935483870968
Specificity: 0.7393767705382436
Threshold: 0.3

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.5715162391370198
AUROC: 0.7684352601420937
AUPRC: 0.6596609927205713
Sensitivity: 0.7074360016253556
Specificity: 0.7024697110904008
Threshold: 0.3

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0035.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5551485044615609
AUROC: 0.8119711230923878
AUPRC: 0.6929331013836794
Sensitivity: 0.7445161290322581
Specificity: 0.7563739376770539
Threshold: 0.24

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.585233063349184
AUROC: 0.7835888509395214
AUPRC: 0.680153000707304
Sensitivity: 0.7257212515237709
Specificity: 0.7150512581547064
Threshold: 0.23

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0036.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.07it/s]

Loss: 0.59342514531953
AUROC: 0.8145033354655944
AUPRC: 0.6985433361793169
Sensitivity: 0.7470967741935484
Specificity: 0.7429178470254958
Threshold: 0.17

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.6376918984471627
AUROC: 0.7855488774935593
AUPRC: 0.6870615873265645
Sensitivity: 0.7147501015847216
Specificity: 0.7220410065237651
Threshold: 0.17

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0037.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5671910762786865
AUROC: 0.8074796673672667
AUPRC: 0.6865932465524954
Sensitivity: 0.7445161290322581
Specificity: 0.7337110481586402
Threshold: 0.22

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.598028153040499
AUROC: 0.7777144990273238
AUPRC: 0.6721383858812077
Sensitivity: 0.7106867127184071
Specificity: 0.7096924510717614
Threshold: 0.22

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0038.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

Loss: 0.5699493067605155
AUROC: 0.8062286393128028
AUPRC: 0.6898170992994509
Sensitivity: 0.7329032258064516
Specificity: 0.7436260623229461
Threshold: 0.21

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.6115994709280302
AUROC: 0.776156219692629
AUPRC: 0.673233595731711
Sensitivity: 0.7025599349857782
Specificity: 0.7141192917054986
Threshold: 0.21

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0039.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5418225186211723
AUROC: 0.8052608973773188
AUPRC: 0.6875161601611246
Sensitivity: 0.7341935483870968
Specificity: 0.7429178470254958
Threshold: 0.28

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.05it/s]

Loss: 0.5686275393895384
AUROC: 0.7770466244523608
AUPRC: 0.6732609200396472
Sensitivity: 0.712312068264933
Specificity: 0.7136533084808947
Threshold: 0.28

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0040.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.03it/s]

Loss: 0.5294744738510677
AUROC: 0.802695787261263
AUPRC: 0.6828664971574568
Sensitivity: 0.7445161290322581
Specificity: 0.7351274787535411
Threshold: 0.3

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.5603052057185263
AUROC: 0.7757717977333637
AUPRC: 0.6705684926405954
Sensitivity: 0.7147501015847216
Specificity: 0.7047996272134203
Threshold: 0.3

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0041.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

Loss: 0.5684827327728271
AUROC: 0.8008754454902678
AUPRC: 0.6811940670722644
Sensitivity: 0.7341935483870968
Specificity: 0.740084985835694
Threshold: 0.22

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.606844001221207
AUROC: 0.7708991866784466
AUPRC: 0.6622585101888001
Sensitivity: 0.7086550182852499
Specificity: 0.7034016775396086
Threshold: 0.22

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0042.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

Loss: 0.5387681143624442
AUROC: 0.7904989490998813
AUPRC: 0.6614565033215394
Sensitivity: 0.7225806451612903
Specificity: 0.7337110481586402
Threshold: 0.32

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.06it/s]

Loss: 0.5680583552369531
AUROC: 0.7625166010074024
AUPRC: 0.6472575073859037
Sensitivity: 0.7021535960991467
Specificity: 0.6982758620689655
Threshold: 0.32

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0043.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5835100889205933
AUROC: 0.8109065155807365
AUPRC: 0.6968146093746534
Sensitivity: 0.7367741935483871
Specificity: 0.7478753541076487
Threshold: 0.19

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.07it/s]

Loss: 0.6212569905339547
AUROC: 0.7800931246930211
AUPRC: 0.6792467131004971
Sensitivity: 0.7049979683055668
Specificity: 0.7215750232991612
Threshold: 0.19

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0044.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.03it/s]

Loss: 0.6088284913982663
AUROC: 0.8031947363611441
AUPRC: 0.6849703468101059
Sensitivity: 0.7264516129032258
Specificity: 0.7471671388101983
Threshold: 0.16

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.6648801493476022
AUROC: 0.7713121527137415
AUPRC: 0.6650315564846779
Sensitivity: 0.7013409183258837
Specificity: 0.7103914259086673
Threshold: 0.16

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0045.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.07it/s]

Loss: 0.5572233072349003
AUROC: 0.7952279996344696
AUPRC: 0.6715206105783889
Sensitivity: 0.7341935483870968
Specificity: 0.726628895184136
Threshold: 0.25

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.5900252191525586
AUROC: 0.7656076451544372
AUPRC: 0.6565478338859068
Sensitivity: 0.6976838683462008
Specificity: 0.712022367194781
Threshold: 0.26

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0046.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.08it/s]

Loss: 0.5716327769415719
AUROC: 0.7985077218313077
AUPRC: 0.6738721429128037
Sensitivity: 0.7354838709677419
Specificity: 0.7245042492917847
Threshold: 0.21

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.6110387450119235
AUROC: 0.7673704193621805
AUPRC: 0.6558065879443377
Sensitivity: 0.6956521739130435
Specificity: 0.7129543336439889
Threshold: 0.22

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0047.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5617766686848231
AUROC: 0.7944859727679795
AUPRC: 0.6725414733710708
Sensitivity: 0.7290322580645161
Specificity: 0.7252124645892352
Threshold: 0.23

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.6017588793106798
AUROC: 0.7642914460930686
AUPRC: 0.6542888836266585
Sensitivity: 0.6980902072328322
Specificity: 0.7092264678471575
Threshold: 0.24

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0048.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5835490499223982
AUROC: 0.805112857534497
AUPRC: 0.6893230412971221
Sensitivity: 0.7303225806451613
Specificity: 0.7407932011331445
Threshold: 0.18

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.00it/s]

Loss: 0.6333599045591535
AUROC: 0.7759493106439961
AUPRC: 0.6748884880709327
Sensitivity: 0.7119057293783015
Specificity: 0.7085274930102516
Threshold: 0.18

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0049.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.06it/s]

Loss: 0.543922437940325
AUROC: 0.7914310518139451
AUPRC: 0.6670047380810928
Sensitivity: 0.7290322580645161
Specificity: 0.7237960339943342
Threshold: 0.27

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.02it/s]

Loss: 0.5811180154669959
AUROC: 0.763947402403875
AUPRC: 0.6531659544535098
Sensitivity: 0.7029662738724096
Specificity: 0.7027027027027027
Threshold: 0.28

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0050.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5589352692876544
AUROC: 0.7957698985652929
AUPRC: 0.672134799645191
Sensitivity: 0.7329032258064516
Specificity: 0.7322946175637394
Threshold: 0.23

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.5996047857235063
AUROC: 0.7668475373326219
AUPRC: 0.6564470666656923
Sensitivity: 0.7131247460381959
Specificity: 0.6971109040074557
Threshold: 0.23

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0051.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5998892911842891
AUROC: 0.7972978159554052
AUPRC: 0.6761794106464618
Sensitivity: 0.7354838709677419
Specificity: 0.726628895184136
Threshold: 0.17

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.06it/s]

Loss: 0.6468638114209445
AUROC: 0.7676024642389592
AUPRC: 0.6596352366886208
Sensitivity: 0.6940268183665177
Specificity: 0.7157502329916123
Threshold: 0.18

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0052.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

Loss: 0.5904938408306667
AUROC: 0.7943909348441925
AUPRC: 0.6769464555541472
Sensitivity: 0.7174193548387097
Specificity: 0.7358356940509915
Threshold: 0.2

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.01it/s]

Loss: 0.626921148232694
AUROC: 0.7653639554307212
AUPRC: 0.6570010644232122
Sensitivity: 0.7013409183258837
Specificity: 0.7024697110904008
Threshold: 0.2

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0053.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

Loss: 0.6117460523332868
AUROC: 0.7999086173809741
AUPRC: 0.6830432028058967
Sensitivity: 0.7212903225806452
Specificity: 0.7415014164305949
Threshold: 0.16

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.05it/s]

Loss: 0.6636719661501219
AUROC: 0.769139678708259
AUPRC: 0.6642929080974225
Sensitivity: 0.6960585127996749
Specificity: 0.7157502329916123
Threshold: 0.16

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0054.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.10it/s]

Loss: 0.5560744200434004
AUROC: 0.7962816412318376
AUPRC: 0.6759771134138498
Sensitivity: 0.7225806451612903
Specificity: 0.740084985835694
Threshold: 0.25

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.5885736796091188
AUROC: 0.7671831550756574
AUPRC: 0.6586858145320005
Sensitivity: 0.699715562779358
Specificity: 0.712022367194781
Threshold: 0.25

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0055.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.12it/s]

Loss: 0.575568094423839
AUROC: 0.7938271040848032
AUPRC: 0.675297412476766
Sensitivity: 0.727741935483871
Specificity: 0.7273371104815864
Threshold: 0.21

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.6161271129857819
AUROC: 0.7651117924240709
AUPRC: 0.6558585555387731
Sensitivity: 0.7029662738724096
Specificity: 0.6978098788443616
Threshold: 0.21

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0056.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.09it/s]

Loss: 0.5932718532426017
AUROC: 0.7941898930823358
AUPRC: 0.6761714026733244
Sensitivity: 0.7316129032258064
Specificity: 0.7259206798866855
Threshold: 0.19

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.02it/s]

Loss: 0.6316090141827205
AUROC: 0.7631830081423042
AUPRC: 0.6537447172600768
Sensitivity: 0.7033726127590411
Specificity: 0.6943150046598322
Threshold: 0.19

Intermediate Model:
  ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0057.model
AUROC/AUPRC on Validation Data

100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00,  5.11it/s]

Loss: 0.5482843109539577
AUROC: 0.7936260623229462
AUPRC: 0.6725973359170988
Sensitivity: 0.7290322580645161
Specificity: 0.7372521246458924
Threshold: 0.25

AUROC/AUPRC on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.06it/s]

Loss: 0.5915976748713907
AUROC: 0.7668495728139971
AUPRC: 0.6587494508486212
Sensitivity: 0.7017472572125152
Specificity: 0.7050326188257223
Threshold: 0.25


Plot AUROC/AUPRC for Each Intermediate Model
  Epoch with best Validation Loss:   42, 0.5378
  Epoch with best model Test AUROC:   1, 0.7967

AUROC/AUPRC Plots - Best Model Based on Validation Loss
  Epoch with best Validation Loss:   42, 0.5378
  Best Model Based on Validation Loss:
    ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0042.model

Generate Stats Based on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00,  5.05it/s]

Loss: 0.5680583552369531
AUROC: 0.7625166010074024
AUPRC: 0.6472575073859037
Sensitivity: 0.7021535960991467
Specificity: 0.6982758620689655
Threshold: 0.32

best_model_val_test_auroc: 0.7625166010074024

best_model_val_test_auprc: 0.6472575073859037

AUROC/AUPRC Plots - Best Model Based on Model AUROC
  Epoch with best model Test AUROC:   1, 0.7967
  Best Model Based on Model AUROC:
    ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0001.model

Generate Stats Based on Test Data

100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00,  5.04it/s]

Loss: 0.6902545052316954
AUROC: 0.7967412321876445
AUPRC: 0.7205366976873793
Sensitivity: 0.7326290125965055
Specificity: 0.7134203168685928
Threshold: 0.14

best_model_auroc_test_auroc: 0.7967412321876445

best_model_auroc_test_auprc: 0.7205366976873793

Results (Planned results for Draft submission)

When we complete our experiments, we will build comparison tables that compare a set of measures for each experiment performed. The full set of experiments and measures are listed below.

Results from Final Rubrik

• Table of results (no need to include additional experiments, but main reproducibility result should be included)
• All claims should be supported by experiment results
• Discuss with respect to the hypothesis and results from the original paper
• Experiments beyond the original paper
  • Each experiment should include results and a discussion
• Ablation Study.

Experiments

• ABP only
• ECG only
• EEG only
• ABP + ECG
• ABP + EEG
• ECG + EEG
• ABP + ECG + EEG

Note: each experiment will be repeated with the following time-to-IOH-event durations:

• 3 minutes
• 5 minutes
• 10 minutes
• 15 minutes

Note: the above list of experiments will be performed if there is sufficient time and gpu capability to complete that before the submission deadline. Should we experience any constraints on this front, we will reduce our experimental coverage to the following 4 core experiments that are necessary to measure the hypotheses included at the head of this report:

• ABP only @ 3 minutes
• ABP + ECG @ 3 minutes
• ABP + EEG @ 3 minutes
• ABP + ECG + EEG @ 3 minutes

For additional details please review the "Planned Actions" in the Discussion section of this report.

Measures

• AUROC
• AUPRC
• Sensitivity
• Specificity
• Threshold
• Loss Shrinkage

[ TODO for final report - collect data for all measures listed above. ]

[ TODO for final report - generate ROC and PRC plots for each experiment ]

We are collecting a broad set of measures across each experiment in order to perform a comprehensive comparison of all measures listed across all comparable experiments executed in the original paper. However, our key experimental results will be focused on a subset of these results that address the main experiments defined at the beginning of this notebook.

The key experimental result measures will be as follows:

• For 3 minutes ahead of the predicted IOH event:
  • compare AUROC and AUPRC for ABP only vs ABP+ECG
  • compare AUROC and AUPRC for ABP only vs ABP+EEG
  • compare AUROC and AUPRC for ABP only vs ABP+ECG+EEG

Model comparison

The following table is Table 3 from the original paper which presents the measured values for each signal combination across each of the four temporal predictive categories:

We have not yet completed the execution of the experiments necessary to determine our reproduced model performance in order determine whether our results are accurately representing those of the original paper. These details are expected to be included in the final report.

As of the draft submission, the reported evaluation measures of our model are too good to be true (all measures are 1.0). We suspect that there is data leakage in the dataset splitting process and will address this in time for the final report.

Discussion

Discussion (10) FROM FINAL RUBRIK

• Implications of the experimental results, whether the original paper was reproducible, and if it wasn’t, what factors made it irreproducible
• “What was easy”
• “What was difficult”
• Recommendations to the original authors or others who work in this area for improving reproducibility
• (specific to our group) "I have communicated with Maciej during OH. The draft looks good and I would expect some explanations/analysis on the final report on why you get 1.0 as AUROC."
  • discuss our bug where we were believing we were sampling dozens of different patient samples but were just training the model on the same segments extracted from the same patient sample over and over. so we were massively overfitting our training data for one patient's data, then unwittingly using the same patient data for validation and testing, thus getting perfect classification during inference.

Feasibility of reproduction

Our assessment is that this paper will be reproducible. The outstanding risk is that each experiment can take up to 7 hours to run on hardware within the team (i.e., 7h to run ~70 epochs on a desktop with AMD Ryzen 7 3800X 8-core CPU w/ RTX 2070 SUPER GPU and 32GB RAM). There are a total of 28 experiments (7 different combinations of signal inputs, 4 different time horizons for each combination). Should our team find it not possible to complete the necessary experiments across all of the experiments represented in Table 3 of our selected paper, we will reduce the number of experiments to focus solely on the ones directly related to our hypotheses described in the beginning of this notebook (i.e., reduce the number of combinations of interest to 4: ABP alone, ABP+EEG, ABP+ECG, ABP+ECG+EEG). This will result in a new total of 16 experiments to run.

Planned ablations

Our proposal included a collection of potential ablations to be investigated:

• Remove ResNet skip connection
• Reduce # of residual blocks from 12 to 6
• Reduce # of residual blocks from 12 to 1
• Eliminate dropout from residual block
• Max pooling configuration
  • smaller size/stride
  • eliminate max pooling

Given the amount of time required to conduct each experiment, our team intends to choose only a small number of ablations from this set. Further, we only intend to perform ablation analysis against the best performing signal combination and time horizon from the reproduction experiments. In order words, we intend to perform ablation analysis against the following training combinations, and only against the models trained with data measured 3 minutes prior to an IOH event:

• ABP alone
• ABP + ECG
• ABP + EEG
• ABP + ECG + EEG

Time and GPU resource permitting, we will complete a broader range of experiments. For additional details, please see the section below titled "Plans for next phase".

Nature of reproduced results

Our team intends to address the manner in which the experimental results align with the published results in the paper in the final submission of this report. The amount of time required to complete model training and result analysis during the preparation of the Draft notebook was not sufficient to complete a large number of experiments.

What was easy? What was difficult?

The difficult aspect of the preparation of this draft involved the data preprocessing.

• First, the source data is unlabelled, so our team was responsible for implementing analysis methods for identifying positive (IOH event occurred) and negative (IOH event did not occur) by running a lookahead analysis of our input training set.
• Second, the volume of raw data is in excess of 90GB. A non-trivial amount of compute was required to minify the input data to only include the data tracks of interest to our experiments (i.e., ABP, ECG, and EEG tracks).
• Third, our team found it difficult to trace back to the definition of the jSQI signal quality index referenced in the paper. Multiple references through multiple papers needed to be traversed to understand which variant of the quality index
  • The only available source code related to the signal quality index as referenced by our paper in [5]. Source code was not directly linked from the paper, but the GitHub repository for the corresponding author for reference [5] did result in the identification of MATLAB source code for the signal quality index as described in the referenced paper. That code is available here: https://github.com/cliffordlab/PhysioNet-Cardiovascular-Signal-Toolbox/tree/master/Tools/BP_Tools
  • Our team had insufficient time to port this signal quality index to Python for use in our investigation, or to setup a MATLAB environment in which to assess our source data using the above MATLAB functions, but we expect to complete this as part of our final report.

Suggestions to paper author

The most notable suggestion would be to correct the hyperparameters published in Supplemental Table 1. Specifically, the output size for residual blocks 11 and 12 for the ECG and ABP data sets was 496x6. This is a typo, and should read 469x6. This typo became apparent when operating the size down operation within Residual Block 11 and recognizing the tensor dimensions were misaligned.

Additionally, more explicit references to the signal quality index assessment tools should be added. Our team could not find a reference to the MATLAB source code as described in reference [3], and had to manually discover the GitHub profile for the lab of the corresponding author of reference [3] in order to find MATLAB source that corresponded to the metrics described therein.

Plans for next phase

Our team plans to accomplish the following goals in service of preparing the Final Report:

• Implement the jSQI filter to remove any training data with aberrent signal quality per the threshold defined in our original paper.
• Execute the following experiments:
  • Measure predictive quality of the model trained solely with ABP data at 3 minutes prior to IOH events.
  • Measure predictive quality of the model trained with ABP+ECG data at 3 minutes prior to IOH events.
  • Measure predictive quality of the model trained with ABP+EEG data at 3 minutes prior to IOH events.
  • Measure predictive quality of the model trained with ABP+ECG+EEG data at 3 minutes prior to IOH events.
• Gather our measures for these experiments and perform a comparison against the published results from our selected paper and determine whether or not we are succesfully reproducing the results outlined in the paper.
• Ablation analysis:
  • Execute the following ablation experiments:
    • Repeat the four experiments described above while reducing the numnber of residual blocks in the model from 12 to 6.
• Time- and/or GPU-resource permitting, we will complete the remaining 24 experiments as described in the paper:
  • Measure predictive quality of the model trained solely with ABP data at 5, 10, and 15 minutes prior to IOH events.
  • Measure predictive quality of the model trained with ABP+ECG data at 5, 10, and 15 minutes prior to IOH events.
  • Measure predictive quality of the model trained with ABP+EEG data at 5, 10, and 15 minutes prior to IOH events.
  • Measure predictive quality of the model trained with ABP+ECG+EEG data at 5, 10, and 15 minutes prior to IOH events.
  • Measure predictive quality of the model trained solely with ECG data at 3, 5, 10, and 15 minutes prior to IOH events.
  • Measure predictive quality of the model trained solely with EEG data at 3, 5, 10, and 15 minutes prior to IOH events.
  • Measure predictive quality of the model trained with ECG+EEG data at 3, 5, 10, and 15 minutes prior to IOH events.
  • Additional ablation experiments:
    • For the four core experiments (ABP, ABP+ECG, ABP+EEG, ABP+ECG+EEG each trained on event data occurring 3 minutes prior to IOH events), perform the following ablations:
      • Repeat experiment while eliminating dropout from every residual block
      • Repeat experiment while removing the skip connection from every residual block
      • Repeat the four experiments described above while reducing the numnber of residual blocks in the model from 12 to 1.

References

1. Jo Y-Y, Jang J-H, Kwon J-m, Lee H-C, Jung C-W, Byun S, et al. “Predicting intraoperative hypotension using deep learning with waveforms of arterial blood pressure, electroencephalogram, and electrocardiogram: Retrospective study.” PLoS ONE, (2022) 17(8): e0272055 https://doi.org/10.1371/journal.pone.0272055
2. Hatib, Feras, Zhongping J, Buddi S, Lee C, Settels J, Sibert K, Rhinehart J, Cannesson M “Machine-learning Algorithm to Predict Hypotension Based on High-fidelity Arterial Pressure Waveform Analysis” Anesthesiology (2018) 129:4 https://doi.org/10.1097/ALN.0000000000002300
3. Bao, X., Kumar, S.S., Shah, N.J. et al. "AcumenTM hypotension prediction index guidance for prevention and treatment of hypotension in noncardiac surgery: a prospective, single-arm, multicenter trial." Perioperative Medicine (2024) 13:13 https://doi.org/10.1186/s13741-024-00369-9
4. Lee, HC., Park, Y., Yoon, S.B. et al. VitalDB, a high-fidelity multi-parameter vital signs database in surgical patients. Sci Data 9, 279 (2022). https://doi.org/10.1038/s41597-022-01411-5
5. Li Q., Mark R.G. & Clifford G.D. "Artificial arterial blood pressure artifact models and an evaluation of a robust blood pressure and heart rate estimator." BioMed Eng OnLine. (2009) 8:13. pmid:19586547 https://doi.org/10.1186/1475-925X-8-13
6. Park H-J, "VitalDB Python Example Notebooks" GitHub Repository https://github.com/vitaldb/examples/blob/master/hypotension_art.ipynb

Public GitHub Repo (5)

• Publish your code in a public repository on GitHub and attach the URL in the notebook.
• Make sure your code is documented properly.
  • A README.md file describing the exact steps to run your code is required.
  • Check “ML Code Completeness Checklist” (https://github.com/paperswithcode/releasing-research-code)
  • Check “Best Practices for Reproducibility” (https://www.cs.mcgill.ca/~ksinha4/practices_for_reproducibility/)

Video Presentation (Requirements from Rubrik)

Walkthrough of the notebook, no need to make slides. We expect a well-timed, well-presented presentation. You should clearly explain what the original paper is about (what the general problem is, what the specific approach taken was, and what the results claimed were) and what you encountered when you attempted to reproduce the results. You should use the time given to you and not too much (or too little).

• <= 4 mins
• Explain the general problem clearly
• Explain the specific approach taken in the paper clearly
• Explain reproduction attempts clearly

print(f'All done!')

All done!